From rea-fbsd at codelabs.ru Sat Nov 1 00:02:34 2008 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Sat Nov 1 00:02:42 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081028081154.GQ6808@hoeg.nl> References: <20081028081154.GQ6808@hoeg.nl> Message-ID: Ed, good day. Tue, Oct 28, 2008 at 09:11:54AM +0100, Ed Schouten wrote: > It's nice to hear that the patch didn't break anything on your system. I > hope to receive more reviews, but I think I'll just commit it this > weekend (with small modifications). Just a quick "me too" message: patchset from October, 29th works flawlessly on my systems. Thanks! -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081101/070eff28/attachment.pgp From keramida at ceid.upatras.gr Sat Nov 1 09:13:19 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Sat Nov 1 09:13:27 2008 Subject: includes, configure, /usr/lib vs. /usr/local/lib, and linux coders In-Reply-To: <539c60b90810311230i11460966la7ff35b0093642ec@mail.gmail.com> (Steve Franks's message of "Fri, 31 Oct 2008 12:30:46 -0700") References: <539c60b90810311123w2aa94b8akcd0a5d0fe791885a@mail.gmail.com> <539c60b90810311230i11460966la7ff35b0093642ec@mail.gmail.com> Message-ID: <87r65vl9ur.fsf@kobe.laptop> On Fri, 31 Oct 2008 12:30:46 -0700, "Steve Franks" wrote: > Let's backup. What's the 'right' way to get a bloody linux program > that expects all it's headers in /usr/include to compile on freebsd > where all the headers are in /usr/local/include? That's all I'm > really asking. Specifically, it's looking for libusb & libftdi. If I > just type gmake, it can't find it, but if I manually edit the > Makefiles to add -I/usr/local/include, it can. Obviously, manually > editing the makefiles is *not* the right way to fix it (plus it's > driving me crazy). Then you run `configure' with the `right' environment: env CPPFLAGS='-I/usr/local/include' \ LDFLAGS='-L/usr/local/lib' ./configure The `--includedir' and `--libdir' options are *not* meant to be useful to the developer that uses the GNU build tools, but to the person who compiles something for the target host. There are several types of people involved in the `release' of a full program: * The maintainer, who uses `automake', `libtool' and `autoconf' to write portable Makefiles and build tools. * The builder, who compiles the program with specific options. * The packager, who runs `make install' in the build tree, creates a set of installed files, and then packages *some* of these files in a packaging-specific format. These types of people commonly have different constraints in the way they can tweak and twist the GNU build tools, to match their needs. 1. The _maintainer_ of the program is free to set up his environment to include any `CPPFLAGS', `CFLAGS' or `LDFLAGS' they find useful. For example, if they have an experimental third-party library installed in a custom location they can use: export CPPFLAGS='-I/opt/foolib/include' LDFLAGS='-L/opt/foolib/lib' ./configure --prefix='/var/tmp/myprog' This way `configure' will emit Makefiles that try to use the third-party library from `/opt/foolib' instead of the system-default location for header files and libraries. This may often be a lot easier than waiting until the necessary bits are installed in the ``official'' places at development systems. Developers who want to experiment with a new version of `libfoo', i.e. to integrate it with the rest of a program, can use custom `CPPFLAGS' and `LDFLAGS' while everyone else is merrily going along with the ``standard'' version of the `libfoo' library. 2. The _builder_ may be constrained in the sets of options he can pass to the `CFLAGS'. He is, after all, testing how a pristine, clean version of the program can build in what is defined as the ``official release'' environment. He may be allowed to tinker with include paths and library paths, but it is often safer to wrap the build process in scripts and tools that yield a repeatable, verifiable build process. This may preclude the use of options like `-I/custom/hdr/path' and `-L/custom/lib/path'. The builder of a program may not be an actual person, but a cron job or another automated process, i.e. a `nightly build' script that runs a clean build in a pristine environment, verifies that it can all complete without errors, and then emails one or more reports. When the builder _is_ a real person, he may be sharing roles with the third type of person involved in the build life of a program that uses the GNU build tools: the packaging person. 3. The _packager_ is someone who runs `make install', to produce the final program distribution and then bundles parts of or even all the files that are produced by the usual `install' target of GNU tools. The installation of all the files may be done in the default installation `prefix', but it may also be redirected to a staging area by setting `DESTDIR' in the environment: mkdir /var/tmp/proto env DESTDIR=/var/tmp/proto make install Depending on the type of the target system, and on particular needs of the packaging person, there may be cases where certain files have to be installed in a `non-standard' location, or in a location that was not foreseen by the original maintainer. In that case, the packager can use the `--libdir' and `--includedir' options to change the final, installed location of the relevant bits. A typical example is the case of Solaris systems, where libraries may be installed in `/usr/lib/64' for 64-bit architectures. When a packager prepares installation images for these architectures, he can build the program with: ./configure --prefix='/opt/foo' --libdir='$prefix/lib/64' All this is a pretty long-winded way of saying: The `--includedir' and `--libdir' options are not really something that is meant to be a convenience option for the _maintainer_ of a program,, the person who writes the code. They are meant to be useful tools for the _packager_ of the program, the person who builds and prepares the final, install-able images. If you are the maintainer, who writes the code of a program, and you find yourself in a position where you need to use the `--libdir' and the `--includedir' options, then YOU ARE DOING IT WRONG. and, as you have probably guessed by now: The Linux people of the original post are wrong. The GNU build tools are a relatively good set of tools to automate a lot of the mundane details of setting up a build system. It is unfortunate that many people who use them have absolutely no clue of what they are using, and they think that slapping a bunch of copy-paste snippets from Google searches in a `configure.in' script will magically turn any odd mess into a clean release process. Alas, Linux newbies are very often exactly _this_ sort of person :/ From max at love2party.net Sat Nov 1 13:14:47 2008 From: max at love2party.net (Max Laier) Date: Sat Nov 1 13:14:54 2008 Subject: du -A / -B options [Re: zfs quota question] In-Reply-To: <200810071314.45922.max@love2party.net> References: <200810071314.45922.max@love2party.net> Message-ID: <200811012114.43137.max@love2party.net> Hi, a thread on freebsd-stable@ [1] about problems with du(1) and compressed zfs filesystems got me looking for a possible solution. Attached is a diff for du(1) that adds two new options: -A to display the apparent size of the file instead of the used blocks. -B bsize to specify a custom blocksize. In particular one <512byte The GNU du(1) has --apparent-size for -A, but we don't like long options. That's not to say that it couldn't be added for script compat. -B is probably not that interesting, but it can be helpful and came for free. Any objections against the general concept? It's rather complicated to get the apparent size of a directory hierarchy without scripting. I often wonder if some hierarchy will fit on a CD/DVD and compressed zfs makes this really difficult. As for the code, I know that there are a couple of style(9) errors in there - mostly because the lines already exceeded 80 chars before my changes and I plan to clean that up before I commit - should there be enough support for the change itself. Thoughts? [1] http://lists.freebsd.org/pipermail/freebsd-stable/2008-October/045698.html -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News -------------- next part -------------- Index: du.1 =================================================================== --- du.1 (revision 184513) +++ du.1 (working copy) @@ -40,11 +40,12 @@ .Nd display disk usage statistics .Sh SYNOPSIS .Nm +.Op Fl A .Op Fl H | L | P .Op Fl a | s | d Ar depth .Op Fl c .Op Fl l -.Op Fl h | k | m +.Op Fl h | k | m | B Ar blocksize .Op Fl n .Op Fl x .Op Fl I Ar mask @@ -60,6 +61,14 @@ .Pp The options are as follows: .Bl -tag -width indent +.It Fl A +Display the apparent size instead of the diskusage. +This can be helpful to find sparse files and when operating on +compressed volumes. +.It Fl B Ar blocksize +Display block counts in +.Ar blocksize +byte blocks. .It Fl H Symbolic links on the command line are followed, symbolic links in file hierarchies are not followed. @@ -136,14 +145,14 @@ If the environment variable .Ev BLOCKSIZE is set, and the -.Fl k -option is not specified, the block counts will be displayed in units of that +.Fl k, m, h, B +options are not specified, the block counts will be displayed in units of that size block. If .Ev BLOCKSIZE is not set, and the -.Fl k -option is not specified, the block counts will be displayed in 512-byte blocks. +.Fl k, m, h, B +options are not specified, the block counts will be displayed in 512-byte blocks. .El .Sh SEE ALSO .Xr df 1 , Index: du.c =================================================================== --- du.c (revision 184513) +++ du.c (working copy) @@ -86,27 +86,39 @@ FTS *fts; FTSENT *p; off_t savednumber = 0; - long blocksize; + long blocksize = 0; int ftsoptions; int listall; int depth; int Hflag, Lflag, Pflag, aflag, sflag, dflag, cflag; - int hflag, lflag, ch, notused, rval; + int Aflag, hflag, lflag, ch, notused, rval; char **save; static char dot[] = "."; setlocale(LC_ALL, ""); Hflag = Lflag = Pflag = aflag = sflag = dflag = cflag = hflag = - lflag = 0; + Aflag = lflag = 0; save = argv; ftsoptions = 0; depth = INT_MAX; SLIST_INIT(&ignores); - while ((ch = getopt(argc, argv, "HI:LPasd:chklmnrx")) != -1) + while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrx")) != -1) switch (ch) { + case 'A': + Aflag = 1; + if (blocksize == 0) + blocksize = 1; + break; + case 'B': + blocksize = atoi(optarg); + if (errno == ERANGE || blocksize < 0) { + warnx("invalid argument to option B: %s", optarg); + usage(); + } + break; case 'H': Hflag = 1; break; @@ -142,23 +154,18 @@ cflag = 1; break; case 'h': - if (setenv("BLOCKSIZE", "512", 1) == -1) - warn( - "setenv: cannot set BLOCKSIZE=512"); hflag = 1; break; case 'k': hflag = 0; - if (setenv("BLOCKSIZE", "1024", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1024"); + blocksize = 1024; break; case 'l': lflag = 1; break; case 'm': hflag = 0; - if (setenv("BLOCKSIZE", "1048576", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1048576"); + blocksize = 1048576; break; case 'n': nodumpflag = 1; @@ -222,8 +229,8 @@ argv[1] = NULL; } - (void) getbsize(¬used, &blocksize); - blocksize /= 512; + if (blocksize == 0) + (void) getbsize(¬used, &blocksize); rval = 0; @@ -241,15 +248,21 @@ break; p->fts_parent->fts_bignum += - p->fts_bignum += p->fts_statp->st_blocks; + p->fts_bignum += Aflag ? + p->fts_statp->st_size : + p->fts_statp->st_blocks; if (p->fts_level <= depth) { if (hflag) { - (void) prthumanval(howmany(p->fts_bignum, blocksize)); + (void) prthumanval(Aflag ? + p->fts_bignum : + p->fts_bignum * DEV_BSIZE); (void) printf("\t%s\n", p->fts_path); } else { (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_bignum, blocksize), + (intmax_t)howmany(Aflag ? + p->fts_bignum : + p->fts_bignum * DEV_BSIZE, blocksize), p->fts_path); } } @@ -272,17 +285,23 @@ if (listall || p->fts_level == 0) { if (hflag) { - (void) prthumanval(howmany(p->fts_statp->st_blocks, - blocksize)); + (void) prthumanval(Aflag ? + p->fts_statp->st_size : + p->fts_statp->st_blocks * DEV_BSIZE); (void) printf("\t%s\n", p->fts_path); } else { (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_statp->st_blocks, blocksize), - p->fts_path); + (intmax_t)howmany( Aflag ? + p->fts_statp->st_size : + p->fts_statp->st_blocks * + DEV_BSIZE, blocksize), + p->fts_path); } } - p->fts_parent->fts_bignum += p->fts_statp->st_blocks; + p->fts_parent->fts_bignum += Aflag ? + p->fts_statp->st_size : + p->fts_statp->st_blocks; } savednumber = p->fts_parent->fts_bignum; } @@ -441,8 +460,6 @@ { char buf[5]; - bytes *= DEV_BSIZE; - humanize_number(buf, sizeof(buf), bytes, "", HN_AUTOSCALE, HN_B | HN_NOSPACE | HN_DECIMAL); @@ -453,8 +470,9 @@ usage(void) { (void)fprintf(stderr, - "usage: du [-H | -L | -P] [-a | -s | -d depth] [-c] " - "[-l] [-h | -k | -m] [-n] [-x] [-I mask] [file ...]\n"); + "usage: du [-A] [-H | -L | -P] [-a | -s | -d depth] [-c] " + "[-l] [-h | -k | -m | -B bsize] [-n] [-x] [-I mask] " + "[file ...]\n"); exit(EX_USAGE); } From freebsd.dev at gmail.com Sat Nov 1 17:36:51 2008 From: freebsd.dev at gmail.com (Biks N) Date: Sat Nov 1 17:37:00 2008 Subject: Need to optimize cutstom kernel hacks Message-ID: <50cd4e5f0811011708t1f4fc665u592602d5085a6d56@mail.gmail.com> Hi, To get started with FreeBSD kernel, I have been working on IP Packet compression. After numerous crashes and failures now everything looks good and stable. I am using kernel zlib routines to compress payload. However I think my implementation is not efficient at all. Here are the steps I am doing for compression: 1. copy Payload to empty buffer using m_copydata() function 2. call deflateInit2 () for deflate initialization 3. call deflate() for actual compression 4. copy the compressed data in buffer back to Payload I have to go through all above 4 steps for each packet! I think it will be lot faster and efficient if: * Somehow get away with deflateinit2() each time for each packet. * I can get to Payload pointer without using m_copydata() so that I don't need to copy data back and forth. Looking for your valuable suggestions and tips :) Bikrant From keramida at freebsd.org Sat Nov 1 17:46:58 2008 From: keramida at freebsd.org (Giorgos Keramidas) Date: Sat Nov 1 17:47:04 2008 Subject: du -A / -B options [Re: zfs quota question] References: <200810071314.45922.max@love2party.net> <200811012114.43137.max@love2party.net> Message-ID: <87zlkjvunk.fsf@kobe.laptop> On Sat, 1 Nov 2008 21:14:42 +0100, Max Laier wrote: > Hi, > a thread on freebsd-stable@ [1] about problems with du(1) and > compressed zfs filesystems got me looking for a possible solution. > Attached is a diff for du(1) that adds two new options: > > -A to display the apparent size of the file instead of the used blocks. > -B bsize to specify a custom blocksize. In particular one <512byte That's nice :) % Index: du.1 % =================================================================== % --- du.1 (revision 184513) % +++ du.1 (working copy) % @@ -60,6 +61,14 @@ % .Pp % The options are as follows: % .Bl -tag -width indent % +.It Fl A % +Display the apparent size instead of the diskusage. 'disk usage' should be two words here. % +This can be helpful to find sparse files and when operating on % +compressed volumes. ``This can be helpful when operating on sparse files or compressed volumes'' maybe? The fact that a difference between plain `du' and `du -A' output may be a nice hack to find sparse files is a cool side-effect, but that's not really the main driving force behind the change, right? :) % Index: du.c % =================================================================== % --- du.c (revision 184513) % +++ du.c (working copy) % @@ -86,27 +86,39 @@ % FTS *fts; % FTSENT *p; % off_t savednumber = 0; % - long blocksize; % + long blocksize = 0; savednumber is one of the existing style bugs. We shouldn't copy it, but initialize `blocksize' further down, near `depth = INT_MAX' and the rest of the initializations. > Thoughts? FWIW, I like the new options :) From khomatapac at gmail.com Sun Nov 2 02:36:51 2008 From: khomatapac at gmail.com (Tapac Khoma) Date: Sun Nov 2 04:10:51 2008 Subject: FreeBSD development Message-ID: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> Hello! I use FreeBSD 6.1 as my desktop system. I like it very much. But there are some features I want my system have. I began to investigate of kernel structure and tried to write some devices drivers. So I want to know can I help for you in development of my favor system. And where I can find useful information about writing device drivers under FreeBSD. I will be wait for your answer. Thank you for attention. Taras Khoma From rpaulo at fnop.net Sun Nov 2 04:23:27 2008 From: rpaulo at fnop.net (Rui Paulo) Date: Sun Nov 2 05:18:50 2008 Subject: FreeBSD development In-Reply-To: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> References: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> Message-ID: <8DE1D793-FA67-42D4-ACB9-36ECC41A5A39@fnop.net> On 2 Nov 2008, at 10:11, Tapac Khoma wrote: > Hello! > > I use FreeBSD 6.1 as my desktop system. I like it very much. But > there are > some features I want my system have. > I began to investigate of kernel structure and tried to write some > devices > drivers. > So I want to know can I help for you in development of my favor > system. And > where I can find useful information about writing device drivers under > FreeBSD. Some pointers: http://www.freebsd.org/projects/index.html http://www.freebsd.org/doc/en/books/developers-handbook/ http://www.freebsd.org/doc/en/books/arch-handbook/ Regards, -- Rui Paulo From freebsd-listen at fabiankeil.de Sun Nov 2 05:35:19 2008 From: freebsd-listen at fabiankeil.de (Fabian Keil) Date: Sun Nov 2 05:35:25 2008 Subject: problems obtaining kernel dump In-Reply-To: <20081026060906.GA66894@underworld.novel.ru> References: <20081026060906.GA66894@underworld.novel.ru> Message-ID: <20081102142026.3d0057f0@fabiankeil.de> Roman Bogorodskiy wrote: > I'm having a problem obtaining kernel dump. The box has 512Mb of RAM. > > In rc.conf I have: > > dumpdev="/dev/ad4s1b" > dumpdir="/var/crash" > > swapinfo -h gives the following: > > Device 1K-blocks Used Avail Capacity > /dev/ad4s1b 1048576 0B 1.0G 0% > > /var/crash directory exists, and the root partition where it is placed > has enough space as well: > > /dev/ad4s1a 989M 350M 560M 38% / > > The box runs fresh FreeBSD/i386 -CURRENT. > > So, I do swapoff and then perform actions to reproduce the crash and it > breaks me into ddb prompt. > > I do: > > call doadump > continue > > it reboots, and when I run "savecore -v /var/crash /dev/ad4s1b" it > prints: > > unable to open bounds file, using 0 > checking for kernel dump on device /dev/ad4s1b > mediasize = 1073741824 > sectorsize = 512 > magic mismatch on last dump header on /dev/ad4s1b > savecore: no dumps found > > The same happens when I do 'panic' instead of 'call doadump' like > handbook suggests. What am I doing wrong? I googled for similar > problems, found some mail threads but didn't find meaningful advises > though. I'm seeing a similar problem with an amd64 system on a SATA disk running CURRENT from 20080727 (can't update as I'm using Pawel's ZFS patch and upgraded most of the zpools). Calling panic leads to an instant reboot, nothing seems to be dumped. Fabian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081102/d85923c0/signature.pgp From patfbsd at davenulle.org Sun Nov 2 07:06:32 2008 From: patfbsd at davenulle.org (Patrick =?ISO-8859-15?Q?Lamaizi=E8re?=) Date: Sun Nov 2 07:06:39 2008 Subject: FreeBSD development In-Reply-To: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> References: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> Message-ID: <20081102154755.1504da7a@baby-jane-lamaiziere-net.local> Le Sun, 2 Nov 2008 12:11:19 +0200, "Tapac Khoma" a ?crit : > Hello! > > I use FreeBSD 6.1 as my desktop system. I like it very much. But > there are some features I want my system have. > I began to investigate of kernel structure and tried to write some > devices drivers. > So I want to know can I help for you in development of my favor > system. And where I can find useful information about writing device > drivers under FreeBSD. I've found this paper quite useful. http://caia.swin.edu.au/reports/070622A/CAIA-TR-070622A.pdf Regards. From shilp.kamal at yahoo.com Sun Nov 2 11:48:40 2008 From: shilp.kamal at yahoo.com (Kamlesh Patel) Date: Sun Nov 2 13:48:40 2008 Subject: Virtual Memory Improvement Algorithm Message-ID: <781226.70062.qm@web45407.mail.sp1.yahoo.com> Hi friends, My name is Kamlesh and i am working on freeBSD Virtual Memory Improvement Algorithm as my Master Project at California State University Sacramento. Could any one help me in Virtual Memory free space management: Splay tree to Radix tree? Thanks Kamlesh From talon at lpthe.jussieu.fr Sun Nov 2 14:40:06 2008 From: talon at lpthe.jussieu.fr (Michel Talon) Date: Sun Nov 2 14:56:43 2008 Subject: Virtual Memory Improvement Algorithm Message-ID: <20081102224001.GA28831@lpthe.jussieu.fr> Kamlesh Patel wrote: > My name is Kamlesh and i am working on freeBSD Virtual Memory > Improvement Algorithm as my Master Project at California State > University Sacramento. > > Could any one help me in Virtual Memory free space management: Splay > tree to Radix tree? You are a late comer, see: http://wiki.freebsd.org/MayurShardul/VM_Algorithm_Improvement -- Michel TALON From julian at elischer.org Sun Nov 2 16:42:40 2008 From: julian at elischer.org (Julian Elischer) Date: Sun Nov 2 16:42:47 2008 Subject: Virtual Memory Improvement Algorithm In-Reply-To: <781226.70062.qm@web45407.mail.sp1.yahoo.com> References: <781226.70062.qm@web45407.mail.sp1.yahoo.com> Message-ID: <490E424D.4010509@elischer.org> Kamlesh Patel wrote: > Hi friends, > > My name is Kamlesh and i am working on freeBSD Virtual Memory Improvement > Algorithm as my Master Project at California State University Sacramento. > > Could any one help me in Virtual Memory free space management: Splay tree > to Radix tree? > > > Thanks > Kamlesh I suggest you talk with professor Cox (alc@freebsd.org) as he is the VM maintainer. > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From yuri at rawbw.com Mon Nov 3 01:32:16 2008 From: yuri at rawbw.com (Yuri) Date: Mon Nov 3 01:32:23 2008 Subject: How to troubleshoot wireless problem? Message-ID: <490EC51E.4070403@rawbw.com> My FreeBSD is able to connect to WiFi WEP network 95% of time. But 5% of time either ifconfig shown "not associated" status or dhclient fails to get IP address or name resolution isn't functioning after DHCP exchange is complete. Signal strength is -80:-95. After a while I am able to connect again. This happens with both ath0 and ral0 devices. In the meantime Linux box sitting next to FreeBSD box has no problem connecting to the same WiFi network at all times. This makes me think that something in FreeBSD isn't working right. How can I troubleshoot this problem? How stable is support for wireless networking in FreeBSD? I use this command line: ifconfig wepmode on ssid my-ssid weptxkey 1 wepkey 0xMYKEY dhcp on 7.1-PRERELEASE Yuri From onemda at gmail.com Mon Nov 3 05:11:02 2008 From: onemda at gmail.com (Paul B. Mahol) Date: Mon Nov 3 05:11:14 2008 Subject: How to troubleshoot wireless problem? In-Reply-To: <490EC51E.4070403@rawbw.com> References: <490EC51E.4070403@rawbw.com> Message-ID: <3a142e750811030446g47d8658cgd6d3cd0daba19c58@mail.gmail.com> On 11/3/08, Yuri wrote: > My FreeBSD is able to connect to WiFi WEP network 95% of time. > But 5% of time either ifconfig shown "not associated" status or dhclient > fails to get IP address or name resolution isn't functioning > after DHCP exchange is complete. Signal strength is -80:-95. > After a while I am able to connect again. > This happens with both ath0 and ral0 devices. > > In the meantime Linux box sitting next to FreeBSD box has > no problem connecting to the same WiFi network at all times. > > This makes me think that something in FreeBSD isn't working right. > > How can I troubleshoot this problem? > How stable is support for wireless networking in FreeBSD? > > I use this command line: > ifconfig wepmode on ssid my-ssid weptxkey 1 wepkey 0xMYKEY dhcp > on 7.1-PRERELEASE In wlan(4) it is explained in detail how to enable debuging. Also every driver have its own ways to toggle debug options. Note that wlan and driver modules must be compiled with debug enabled(at least it works in that way on CURRENT). Look in NOTES for explanation. From patfbsd at davenulle.org Mon Nov 3 12:50:25 2008 From: patfbsd at davenulle.org (Patrick =?ISO-8859-15?Q?Lamaizi=E8re?=) Date: Mon Nov 3 12:50:32 2008 Subject: problem with bus_dmamap_load_uio Message-ID: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> Hello, (8-Current/i386) I've got a problem with bus_dmamap_load_uio(9). I want to use it to map an uio for DMA with the Geode security block (glxsb(4)). It works, i can make milions of crypto operations with cryptotest. But when (i guess, i'm not sure) the free memory becomes low, bus_dmamap_load_uio() fails with errno == EFBIG. I can reproduce the problem by running "periodic daily", "dd if=/dev/zero of=foo", ... By sample i run into problems with Mem: 33M Active, 310M Inact, 82M Wired, 60M Buf, 69M Free. When it fails the uio is : uio_segflg = UIO_SYSSPACE uio_iovcnt = 1, totlen = 16384 (the total length for iov) uio_resid = 16384 (i've got some failure with size between 4000 and 16384) I don't understand why bus_dmamap_load_uio() cannot load the uio. Also when it fails, i copy the uio into a buffer and then a bus_dmamap_load() of the buffer always works. The dma tag is allocated with: bus_dma_tag_create(bus_get_dma_tag(sc->sc_dev),/* parent */ 16, 0, /* alignments, bounds */ BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ 16384, /* maxsize */ 1, /* nsegments */ 16384, /* maxsegsize */ BUS_DMA_ALLOCNOW, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->sc_dmat); The dma map is created just before the bus_dmamap_load_uio() and destroyed later. (source : http://user.lamaiziere.net/patrick/glxsb.c , see glxsb_crypto_encdec() ). I'm trying to understand and fix this problem for several days, so any idea will be very cool... thanks, regards. From stevefranks at ieee.org Mon Nov 3 14:12:35 2008 From: stevefranks at ieee.org (Steve Franks) Date: Mon Nov 3 14:12:42 2008 Subject: includes, configure, /usr/lib vs. /usr/local/lib, and linux coders In-Reply-To: <87r65vl9ur.fsf@kobe.laptop> References: <539c60b90810311123w2aa94b8akcd0a5d0fe791885a@mail.gmail.com> <539c60b90810311230i11460966la7ff35b0093642ec@mail.gmail.com> <87r65vl9ur.fsf@kobe.laptop> Message-ID: <539c60b90811031412y35e2599cneca5912ada43b081@mail.gmail.com> That's alot of good info. It should go in the porter's handbook, maybe... So, if I'm looking to make a port, which one of those people should I be acting as? Maintainer? That's FreeBSD-port-terminology you are using, correct? Steve On Sat, Nov 1, 2008 at 8:55 AM, Giorgos Keramidas wrote: > On Fri, 31 Oct 2008 12:30:46 -0700, "Steve Franks" wrote: >> Let's backup. What's the 'right' way to get a bloody linux program >> that expects all it's headers in /usr/include to compile on freebsd >> where all the headers are in /usr/local/include? That's all I'm >> really asking. Specifically, it's looking for libusb & libftdi. If I >> just type gmake, it can't find it, but if I manually edit the >> Makefiles to add -I/usr/local/include, it can. Obviously, manually >> editing the makefiles is *not* the right way to fix it (plus it's >> driving me crazy). > > Then you run `configure' with the `right' environment: > > env CPPFLAGS='-I/usr/local/include' \ > LDFLAGS='-L/usr/local/lib' ./configure > > The `--includedir' and `--libdir' options are *not* meant to be useful > to the developer that uses the GNU build tools, but to the person who > compiles something for the target host. > > There are several types of people involved in the `release' of a full > program: > > * The maintainer, who uses `automake', `libtool' and `autoconf' to > write portable Makefiles and build tools. > > * The builder, who compiles the program with specific options. > > * The packager, who runs `make install' in the build tree, creates a > set of installed files, and then packages *some* of these files in > a packaging-specific format. > > These types of people commonly have different constraints in the way > they can tweak and twist the GNU build tools, to match their needs. > > 1. The _maintainer_ of the program is free to set up his environment to > include any `CPPFLAGS', `CFLAGS' or `LDFLAGS' they find useful. For > example, if they have an experimental third-party library installed > in a custom location they can use: > > export CPPFLAGS='-I/opt/foolib/include' LDFLAGS='-L/opt/foolib/lib' > ./configure --prefix='/var/tmp/myprog' > > This way `configure' will emit Makefiles that try to use the > third-party library from `/opt/foolib' instead of the system-default > location for header files and libraries. > > This may often be a lot easier than waiting until the necessary bits > are installed in the ``official'' places at development systems. > Developers who want to experiment with a new version of `libfoo', > i.e. to integrate it with the rest of a program, can use custom > `CPPFLAGS' and `LDFLAGS' while everyone else is merrily going along > with the ``standard'' version of the `libfoo' library. > > 2. The _builder_ may be constrained in the sets of options he can pass > to the `CFLAGS'. He is, after all, testing how a pristine, clean > version of the program can build in what is defined as the ``official > release'' environment. > > He may be allowed to tinker with include paths and library paths, but > it is often safer to wrap the build process in scripts and tools that > yield a repeatable, verifiable build process. This may preclude the > use of options like `-I/custom/hdr/path' and `-L/custom/lib/path'. > > The builder of a program may not be an actual person, but a cron job > or another automated process, i.e. a `nightly build' script that runs > a clean build in a pristine environment, verifies that it can all > complete without errors, and then emails one or more reports. > > When the builder _is_ a real person, he may be sharing roles with the > third type of person involved in the build life of a program that > uses the GNU build tools: the packaging person. > > 3. The _packager_ is someone who runs `make install', to produce the > final program distribution and then bundles parts of or even all the > files that are produced by the usual `install' target of GNU tools. > The installation of all the files may be done in the default > installation `prefix', but it may also be redirected to a staging > area by setting `DESTDIR' in the environment: > > mkdir /var/tmp/proto > env DESTDIR=/var/tmp/proto make install > > Depending on the type of the target system, and on particular needs > of the packaging person, there may be cases where certain files have > to be installed in a `non-standard' location, or in a location that > was not foreseen by the original maintainer. In that case, the > packager can use the `--libdir' and `--includedir' options to change > the final, installed location of the relevant bits. > > A typical example is the case of Solaris systems, where libraries may > be installed in `/usr/lib/64' for 64-bit architectures. When a > packager prepares installation images for these architectures, he can > build the program with: > > ./configure --prefix='/opt/foo' --libdir='$prefix/lib/64' > > All this is a pretty long-winded way of saying: > > The `--includedir' and `--libdir' options are not really something > that is meant to be a convenience option for the _maintainer_ of a > program,, the person who writes the code. They are meant to be > useful tools for the _packager_ of the program, the person who > builds and prepares the final, install-able images. > > If you are the maintainer, who writes the code of a program, and you > find yourself in a position where you need to use the `--libdir' and > the `--includedir' options, then YOU ARE DOING IT WRONG. > > and, as you have probably guessed by now: > > The Linux people of the original post are wrong. > > The GNU build tools are a relatively good set of tools to automate a lot > of the mundane details of setting up a build system. It is unfortunate > that many people who use them have absolutely no clue of what they are > using, and they think that slapping a bunch of copy-paste snippets from > Google searches in a `configure.in' script will magically turn any odd > mess into a clean release process. Alas, Linux newbies are very often > exactly _this_ sort of person :/ > > From jhb at freebsd.org Mon Nov 3 14:51:25 2008 From: jhb at freebsd.org (John Baldwin) Date: Mon Nov 3 14:51:31 2008 Subject: problem with bus_dmamap_load_uio In-Reply-To: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> References: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> Message-ID: <200811031849.57232.jhb@freebsd.org> On Monday 03 November 2008 03:50:21 pm Patrick Lamaizi?re wrote: > Hello, > > (8-Current/i386) > > I've got a problem with bus_dmamap_load_uio(9). I want to use it to > map an uio for DMA with the Geode security block (glxsb(4)). > > It works, i can make milions of crypto operations with cryptotest. > > But when (i guess, i'm not sure) the free memory becomes low, > bus_dmamap_load_uio() fails with errno == EFBIG. > > I can reproduce the problem by running "periodic daily", "dd > if=/dev/zero of=foo", ... > > By sample i run into problems with Mem: 33M Active, 310M Inact, 82M > Wired, 60M Buf, 69M Free. > > When it fails the uio is : > uio_segflg = UIO_SYSSPACE > uio_iovcnt = 1, > totlen = 16384 (the total length for iov) > uio_resid = 16384 > > (i've got some failure with size between 4000 and 16384) > > I don't understand why bus_dmamap_load_uio() cannot load the uio. Also > when it fails, i copy the uio into a buffer and then a > bus_dmamap_load() of the buffer always works. > > The dma tag is allocated with: > > bus_dma_tag_create(bus_get_dma_tag(sc->sc_dev),/* parent */ > 16, 0, /* alignments, bounds */ > BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ > BUS_SPACE_MAXADDR, /* highaddr */ > NULL, NULL, /* filter, filterarg */ > 16384, /* maxsize */ > 1, /* nsegments */ > 16384, /* maxsegsize */ > BUS_DMA_ALLOCNOW, /* flags */ > NULL, NULL, /* lockfunc, lockarg */ > &sc->sc_dmat); > > The dma map is created just before the bus_dmamap_load_uio() and > destroyed later. > > (source : http://user.lamaiziere.net/patrick/glxsb.c , see > glxsb_crypto_encdec() ). > > I'm trying to understand and fix this problem for several days, so any > idea will be very cool... Your dma tag only allows a single scatter/gather entry (nsegments). What is happening is that under memory pressure, the virtual buffer in userland is not built from physically contiguous pages, so you would need multiple s/g entries to describe the buffer. Hence EFBIG. If your hardware can handle multiple s/g entries, then just increase 'nsegments' and handle the multiple segments in your callback routine. -- John Baldwin From patfbsd at davenulle.org Tue Nov 4 04:34:14 2008 From: patfbsd at davenulle.org (Patrick =?ISO-8859-15?Q?Lamaizi=E8re?=) Date: Tue Nov 4 04:34:22 2008 Subject: problem with bus_dmamap_load_uio In-Reply-To: <200811031849.57232.jhb@freebsd.org> References: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> <200811031849.57232.jhb@freebsd.org> Message-ID: <20081104133408.7a010b63@baby-jane-lamaiziere-net.local> Le Mon, 3 Nov 2008 18:49:57 -0500, John Baldwin a ?crit : Hello, > > I don't understand why bus_dmamap_load_uio() cannot load the uio. > > Also when it fails, i copy the uio into a buffer and then a > > bus_dmamap_load() of the buffer always works. > > > > The dma tag is allocated with: > > > > bus_dma_tag_create(bus_get_dma_tag(sc->sc_dev),/* parent */ > > 16, 0, /* alignments, bounds */ > > BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ > > BUS_SPACE_MAXADDR, /* highaddr */ > > NULL, NULL, /* filter, filterarg */ > > 16384, /* maxsize */ > > 1, /* nsegments */ > > 16384, /* maxsegsize */ > > BUS_DMA_ALLOCNOW, /* flags */ > > NULL, NULL, /* lockfunc, lockarg */ > > &sc->sc_dmat); ... > Your dma tag only allows a single scatter/gather entry (nsegments). > What is happening is that under memory pressure, the virtual buffer > in userland is not built from physically contiguous pages, so you > would need multiple s/g entries to describe the buffer. Hence > EFBIG. Thank you John, you save me! > If your hardware can handle multiple s/g entries, then just > increase 'nsegments' and handle the multiple segments in your > callback routine. Unfortunely it does not. Regards. From venkatraman_t at yahoo.com Tue Nov 4 11:04:42 2008 From: venkatraman_t at yahoo.com (Venkatraman T) Date: Tue Nov 4 12:17:02 2008 Subject: Using GCov in the FreeBSD kernel Message-ID: <379172.84312.qm@web34406.mail.mud.yahoo.com> Hi, I am trying to understand what how much effort it would take to get the code coverage in FreeBSD kernel using gcov. My understanding is we have gcov in FreeBSD which gets code coverage on user space programs. The reasons that this can't be used in kernel right now is because: 1) gcov libs cannot use the standard C libs when the kernel code is compiled with the gcov flags 2) There might be some issues with the kernel's access to the file systems for dumping the .gcda files What are the other issues that I need to look at? There is also a kernel patch for Linux (LTP) that allows gcov to get coverage in the kernel. Will it easier to patch FreeBSD gcov libraries to run on the kernel or should I try and modify the Linux kernel patch to work on FreeBSD kernel. I am very new to anything in the kernel and would appreciate any insight on how to go about doing this. Thanks Venkat From ivoras at freebsd.org Tue Nov 4 14:13:41 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Nov 4 14:13:48 2008 Subject: YAVP (Yet Another Vmstat Patch) Message-ID: Hi, I'm working on something that tends to generate a lot of context switches and I don't like the way values in "vmstat 1" are printed practically unbounded, causing wrapping, etc. Here's a patch against -CURRENT that uses humanize_number() to keep the formatting of memory and context switches tabular in the common case: http://people.freebsd.org/~ivoras/diffs/vmstat.c.patch.txt The following people might not like it: * Those who really need to distinguish between 185400 context switches and 185499 context switches * Those who have scripts that parse the output * Those who thought the previous chaotic behaviour is cute :) Example of new output is: procs memory page disks faults cpu r b w avm free flt re pi po fr sr ad4 ad8 in sy cs us sy id 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 2 373K 267K 4 37 59 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 366K 271K 4 37 59 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 370K 267K 4 36 59 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 378K 278K 5 36 60 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 69 366K 268K 4 39 58 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 65 372K 271K 5 41 54 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 71 374K 271K 5 40 55 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 6 375K 276K 4 39 57 Any objections to the patch? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 258 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081104/7eab56e0/signature.pgp From max at love2party.net Tue Nov 4 14:42:52 2008 From: max at love2party.net (Max Laier) Date: Tue Nov 4 14:43:01 2008 Subject: du -A / -B options [Re: zfs quota question] In-Reply-To: <200811012114.43137.max@love2party.net> References: <200810071314.45922.max@love2party.net> <200811012114.43137.max@love2party.net> Message-ID: <200811042342.49827.max@love2party.net> Hi again, On Saturday 01 November 2008 21:14:42 I wrote: > a thread on freebsd-stable@ [1] about problems with du(1) and compressed > zfs filesystems got me looking for a possible solution. Attached is a diff > for du(1) that adds two new options: > > -A to display the apparent size of the file instead of the used blocks. > -B bsize to specify a custom blocksize. In particular one <512byte > > The GNU du(1) has --apparent-size for -A, but we don't like long options. > That's not to say that it couldn't be added for script compat. -B is > probably not that interesting, but it can be helpful and came for free. Attached is an updated patch. This refines the -B option to something more useful (and fixes a bug in the original patch). From the man page: -B blocksize Calculate block counts in blocksize byte blocks. This is differ- ent from the -k, -m options or setting BLOCKSIZE and gives an estimate of how many space the examined file hierachy would require on a filesystem with the given blocksize. Unless in -A mode, blocksize is rounded up to the next multiple of 512. I think this is also very helpful and came for almost free. I made sure to not loose any range in the calculations when not in -A mode. In -A mode and with small -B values you obviously can blast the 2^63bit block counter in struct fts earlier. Please review, test & comment ... I plan to commit this in a few days if I don't hear otherwise. Thanks. > Any objections against the general concept? It's rather complicated to get > the apparent size of a directory hierarchy without scripting. I often > wonder if some hierarchy will fit on a CD/DVD and compressed zfs makes this > really difficult. > > As for the code, I know that there are a couple of style(9) errors in there > - mostly because the lines already exceeded 80 chars before my changes and > I plan to clean that up before I commit - should there be enough support > for the change itself. > > Thoughts? > > [1] > http://lists.freebsd.org/pipermail/freebsd-stable/2008-October/045698.html -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News -------------- next part -------------- Index: du.1 =================================================================== --- du.1 (revision 184563) +++ du.1 (working copy) @@ -40,11 +40,12 @@ .Nd display disk usage statistics .Sh SYNOPSIS .Nm +.Op Fl A .Op Fl H | L | P .Op Fl a | s | d Ar depth .Op Fl c .Op Fl l -.Op Fl h | k | m +.Op Fl h | k | m | B Ar blocksize .Op Fl n .Op Fl x .Op Fl I Ar mask @@ -60,6 +61,25 @@ .Pp The options are as follows: .Bl -tag -width indent +.It Fl A +Display the apparent size instead of the disk usage. +This can be helpful when operating on compressed volumes or sparse files. +.It Fl B Ar blocksize +Calculate block counts in +.Ar blocksize +byte blocks. +This is different from the +.Fl k, m +options or setting +.Ev BLOCKSIZE +and gives an estimate of how many space the examined file hierachy would +require on a filesystem with the given +.Ar blocksize . +Unless in +.Fl A +mode, +.Ar blocksize +is rounded up to the next multiple of 512. .It Fl H Symbolic links on the command line are followed, symbolic links in file hierarchies are not followed. @@ -136,14 +156,19 @@ If the environment variable .Ev BLOCKSIZE is set, and the -.Fl k -option is not specified, the block counts will be displayed in units of that -size block. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in units of +that size block. If .Ev BLOCKSIZE is not set, and the -.Fl k -option is not specified, the block counts will be displayed in 512-byte blocks. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in 512-byte +blocks. .El .Sh SEE ALSO .Xr df 1 , Index: du.c =================================================================== --- du.c (revision 184656) +++ du.c (working copy) @@ -79,14 +79,15 @@ static int ignorep(FTSENT *); static int nodumpflag = 0; +static int Aflag; +static long blocksize, cblocksize; int main(int argc, char *argv[]) { FTS *fts; FTSENT *p; - off_t savednumber; - long blocksize; + off_t savednumber, curblocks; int ftsoptions; int listall; int depth; @@ -98,16 +99,30 @@ setlocale(LC_ALL, ""); Hflag = Lflag = Pflag = aflag = sflag = dflag = cflag = hflag = - lflag = 0; + lflag = Aflag = 0; save = argv; ftsoptions = 0; savednumber = 0; + cblocksize = DEV_BSIZE; + blocksize = 0; depth = INT_MAX; SLIST_INIT(&ignores); - while ((ch = getopt(argc, argv, "HI:LPasd:chklmnrx")) != -1) + while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrx")) != -1) switch (ch) { + case 'A': + Aflag = 1; + break; + case 'B': + errno = 0; + cblocksize = atoi(optarg); + if (errno == ERANGE || cblocksize < 0) { + warnx("invalid argument to option B: %s", + optarg); + usage(); + } + break; case 'H': Hflag = 1; break; @@ -144,22 +159,18 @@ cflag = 1; break; case 'h': - if (setenv("BLOCKSIZE", "512", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=512"); hflag = 1; break; case 'k': hflag = 0; - if (setenv("BLOCKSIZE", "1024", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1024"); + blocksize = 1024; break; case 'l': lflag = 1; break; case 'm': hflag = 0; - if (setenv("BLOCKSIZE", "1048576", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1048576"); + blocksize = 1048576; break; case 'n': nodumpflag = 1; @@ -206,6 +217,9 @@ if (Pflag) ftsoptions |= FTS_PHYSICAL; + if (!Aflag && (cblocksize % DEV_BSIZE) != 0) + cblocksize = howmany(cblocksize, DEV_BSIZE) * DEV_BSIZE; + listall = 0; if (aflag) { @@ -224,9 +238,14 @@ argv[1] = NULL; } - (void)getbsize(¬used, &blocksize); - blocksize /= 512; + if (blocksize == 0) + (void)getbsize(¬used, &blocksize); + if (!Aflag) { + cblocksize /= DEV_BSIZE; + blocksize /= DEV_BSIZE; + } + rval = 0; if ((fts = fts_open(argv, ftsoptions, NULL)) == NULL) @@ -242,17 +261,19 @@ if (ignorep(p)) break; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); p->fts_parent->fts_bignum += p->fts_bignum += - p->fts_statp->st_blocks; + curblocks; if (p->fts_level <= depth) { if (hflag) { - prthumanval(howmany(p->fts_bignum, - blocksize)); + prthumanval(p->fts_bignum); (void)printf("\t%s\n", p->fts_path); } else { (void)printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_bignum, + howmany(p->fts_bignum * cblocksize, blocksize), p->fts_path); } } @@ -273,21 +294,22 @@ linkchk(p)) break; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); + if (listall || p->fts_level == 0) { if (hflag) { - prthumanval(howmany( - p->fts_statp->st_blocks, - blocksize)); + prthumanval(curblocks); (void)printf("\t%s\n", p->fts_path); } else { (void)printf("%jd\t%s\n", - (intmax_t)howmany( - p->fts_statp->st_blocks, - blocksize), p->fts_path); + howmany(curblocks * cblocksize, + blocksize), p->fts_path); } } - p->fts_parent->fts_bignum += p->fts_statp->st_blocks; + p->fts_parent->fts_bignum += curblocks; } savednumber = p->fts_parent->fts_bignum; } @@ -297,11 +319,11 @@ if (cflag) { if (hflag) { - prthumanval(howmany(savednumber, blocksize)); + prthumanval(savednumber); (void)printf("\ttotal\n"); } else { (void)printf("%jd\ttotal\n", (intmax_t)howmany( - savednumber, blocksize)); + savednumber * cblocksize, blocksize)); } } @@ -448,7 +470,9 @@ { char buf[5]; - bytes *= DEV_BSIZE; + bytes *= cblocksize; + if (!Aflag) + bytes *= DEV_BSIZE; humanize_number(buf, sizeof(buf), bytes, "", HN_AUTOSCALE, HN_B | HN_NOSPACE | HN_DECIMAL); @@ -460,8 +484,9 @@ usage(void) { (void)fprintf(stderr, - "usage: du [-H | -L | -P] [-a | -s | -d depth] [-c] " - "[-l] [-h | -k | -m] [-n] [-x] [-I mask] [file ...]\n"); + "usage: du [-A] [-H | -L | -P] [-a | -s | -d depth] [-c] " + "[-l] [-h | -k | -m | -B bsize] [-n] [-x] [-I mask] " + "[file ...]\n"); exit(EX_USAGE); } From keramida at ceid.upatras.gr Tue Nov 4 16:05:56 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Tue Nov 4 16:06:03 2008 Subject: YAVP (Yet Another Vmstat Patch) In-Reply-To: (Ivan Voras's message of "Tue, 04 Nov 2008 23:13:19 +0100") References: Message-ID: <87mygfkpo3.fsf@kobe.laptop> On Tue, 04 Nov 2008 23:13:19 +0100, Ivan Voras wrote: > Hi, > > I'm working on something that tends to generate a lot of context > switches and I don't like the way values in "vmstat 1" are printed > practically unbounded, causing wrapping, etc. > > Here's a patch against -CURRENT that uses humanize_number() to keep the > formatting of memory and context switches tabular in the common case: > > http://people.freebsd.org/~ivoras/diffs/vmstat.c.patch.txt > > The following people might not like it: > > * Those who really need to distinguish between 185400 context switches > and 185499 context switches > * Those who have scripts that parse the output > * Those who thought the previous chaotic behaviour is cute :) > > Example of new output is: > > procs memory page disks faults cpu > r b w avm free flt re pi po fr sr ad4 ad8 in sy cs us sy id > 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 2 373K 267K 4 37 59 > 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 366K 271K 4 37 59 > 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 370K 267K 4 36 59 > 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 378K 278K 5 36 60 > 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 69 366K 268K 4 39 58 > 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 65 372K 271K 5 41 54 > 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 71 374K 271K 5 40 55 > 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 6 375K 276K 4 39 57 > > Any objections to the patch? Can we convince you to make it an option and default to off at first? vmstat's current output format has the useful property that fault columns align nicely when they are ``small enough'', and seeing them misaligned is something that triggers a ``we have considerable load'' double-check. Right now my backup script is running, and I see: : $ vmstat -w 1 : procs memory page disks faults cpu : r b w avm fre flt re pi po fr sr ad0 md0 in sy cs us sy id : 3 0 0 1071M 211M 903 0 0 0 1176 388 0 0 661 16032 5783 12 9 79 : 3 0 0 1071M 210M 1 0 0 0 30 0 74 0 1356 3062 7927 7 15 78 : 0 0 0 1071M 209M 20 0 0 0 32 0 79 0 1509 3139 8629 4 16 80 : 0 0 0 1071M 207M 0 0 0 0 30 0 74 0 1306 2988 7653 4 14 82 : 0 0 0 1071M 206M 0 0 0 0 31 0 63 0 1315 2787 7302 6 17 77 : 1 0 0 1071M 205M 0 0 0 0 30 0 49 0 1787 2786 8750 4 20 76 : 0 0 0 1071M 204M 0 0 0 0 30 0 49 0 1974 2465 9480 5 19 76 : 0 0 0 1071M 202M 0 0 0 0 30 0 89 0 1388 3588 8432 6 16 78 : 0 0 0 1071M 200M 1 0 0 0 30 0 87 0 1358 3338 8199 6 17 77 : 3 0 0 1071M 199M 0 0 0 0 30 0 80 0 1367 4142 8216 9 16 75 : 1 0 0 1071M 198M 0 0 0 0 30 0 76 0 1339 3175 8133 6 18 76 : 1 0 0 1071M 241M 0 0 0 0 11354 0 73 0 1361 3055 7839 8 24 68 : 3 0 0 1057M 253M 0 0 0 0 3426 0 82 0 1398 3548 8273 10 17 73 : 2 0 0 1057M 247M 0 0 0 0 31 0 123 0 1357 4377 8113 4 14 82 : 1 0 0 1057M 240M 36 0 1 0 47 0 150 0 1394 4597 8526 6 16 78 : 1 0 0 1057M 233M 0 0 0 0 30 0 146 0 1298 4496 7970 4 15 81 : ^C Only the first line is misaligned by _one_ character, and I *like* the extra granularity of the rest. Your patched version has a shorter `avm' and `fre' too. Do we really have to trim them? They seem to be doing fine with physmem >= 3 GB. From maksim.yevmenkin at gmail.com Tue Nov 4 16:19:31 2008 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Tue Nov 4 16:19:37 2008 Subject: converting strings from utf8 Message-ID: Hello, can i use wcstombs(3) to convert a string presented in utf8 into current locale? basically i'm looking for something like iconv from ports but included into base system. in other words, would something like this work? char *locale, dst[256]; size_t len; locale = setlocale(LC_CTYPE, ""); if (locale == NULL) /* bail */ len = wcstombs(dst, (wchar_t *) src, sizeof(dst)); if (len < 0) /* bail */ printf("%*.*s\n", len, len, dst); thanks, max From phocking at fsklaw.com Tue Nov 4 16:49:40 2008 From: phocking at fsklaw.com (Phillip Hocking) Date: Tue Nov 4 17:01:15 2008 Subject: Kernel panic on 7.0-RELEASE Message-ID: <4910E9A0.5000708@fsklaw.com> Hello all, I keep getting a kernel panic every Saturday night, so I figured I would go through the core dump. # uname -a FreeBSD xx.fsklaw.com 7.0-RELEASE FreeBSD 7.0-RELEASE #1: Wed Apr 23 08:01:10 PDT 2008 xx@to-new.fsklaw.com:/usr/obj/usr/src/sys/TEST amd64 to# kgdb kernel.symbols /var/crash/vmcore.2 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x250 fault code = supervisor read data, page not present instruction pointer = 0x8:0xffffffff80482a5d stack pointer = 0x10:0xffffffffb184b570 frame pointer = 0x10:0xffffff003f9cf6a0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 37920 (bsdtar) trap number = 12 panic: page fault cpuid = 1 Uptime: 7d2h23m33s Physical memory: 2034 MB Dumping 415 MB: 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16 #0 doadump () at pcpu.h:194 194 __asm __volatile("movq %%gs:0,%0" : "=r" (td)); (kgdb) where #0 doadump () at pcpu.h:194 #1 0x0000000000000004 in ?? () #2 0xffffffff8048e0d9 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #3 0xffffffff8048e4dd in panic (fmt=0x104
) at /usr/src/sys/kern/kern_shutdown.c:563 #4 0xffffffff80786f44 in trap_fatal (frame=0xffffff003f9cf6a0, eva=18446742975518566608) at /usr/src/sys/amd64/amd64/trap.c:724 #5 0xffffffff80787315 in trap_pfault (frame=0xffffffffb184b4c0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:641 #6 0xffffffff80787c58 in trap (frame=0xffffffffb184b4c0) at /usr/src/sys/amd64/amd64/trap.c:410 #7 0xffffffff8076d88e in calltrap () at /usr/src/sys/amd64/amd64/exception.S:169 #8 0xffffffff80482a5d in _mtx_lock_sleep (m=0xffffff009662c410, tid=18446742975265175200, opts=Variable "opts" is not available. ) at /usr/src/sys/kern/kern_mutex.c:335 #9 0xffffffff804f5978 in vfs_vmio_release (bp=0xffffffff9a329220) at /usr/src/sys/kern/vfs_bio.c:1530 #10 0xffffffff804f77f4 in getnewbuf (slpflag=0, slptimeo=0, size=Variable "size" is not available. ) at /usr/src/sys/kern/vfs_bio.c:1847 #11 0xffffffff804f8801 in getblk (vp=0xffffff000b71c5d0, blkno=294994, size=16384, slpflag=0, slptimeo=0, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/vfs_bio.c:2602 #12 0xffffffff806a3d5b in ffs_balloc_ufs2 (vp=0xffffff000b71c5d0, startoffset=Variable "startoffset" is not available. ) at /usr/src/sys/ufs/ffs/ffs_balloc.c:844 #13 0xffffffff806bf6e5 in ffs_write (ap=0xffffffffb184ba10) at /usr/src/sys/ufs/ffs/ffs_vnops.c:720 #14 0xffffffff807e04e5 in VOP_WRITE_APV (vop=0xffffffff80a6b0e0, a=0xffffffffb184ba10) at vnode_if.c:691 #15 0xffffffff8051ae41 in vn_write (fp=0xffffff00284771e0, uio=0xffffffffb184bb00, active_cred=Variable "active_cred" is not available. ) at vnode_if.h:373 #16 0xffffffff804c6468 in dofilewrite (td=0xffffff003f9cf6a0, fd=3, fp=0xffffff00284771e0, auio=0xffffffffb184bb00, offset=Variable "offset" is not available. ) at file.h:254 #17 0xffffffff804c676e in kern_writev (td=0xffffff003f9cf6a0, fd=3, auio=0xffffffffb184bb00) at /usr/src/sys/kern/sys_generic.c:401 #18 0xffffffff804c67ec in write (td=0xffffff009662c410, uap=0xffffff003f9cf6a0) at /usr/src/sys/kern/sys_generic.c:317 #19 0xffffffff80787597 in syscall (frame=0xffffffffb184bc70) at /usr/src/sys/amd64/amd64/trap.c:852 #20 0xffffffff8076da9b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:290 #21 0x0000000800a66d3c in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) Note the uptime, this happens every Saturday for the last three weeks. I have a cronjob that tar/bzips a drive and scps it to another box, but that runs nightly at midnight on the cronjob, and this happens weekly but only on Saturday, same cronjob same data. I figured if it happened again, I would hit all you leet-sauce developers up. Hopefully you all can suggest what to upgrade to or how to further troubleshoot this issue. Thanks a bundle y'all, love the beastie! From keramida at freebsd.org Tue Nov 4 18:31:59 2008 From: keramida at freebsd.org (Giorgos Keramidas) Date: Tue Nov 4 18:32:18 2008 Subject: du -A / -B options [Re: zfs quota question] In-Reply-To: <200811042342.49827.max@love2party.net> (Max Laier's message of "Tue, 4 Nov 2008 23:42:49 +0100") References: <200810071314.45922.max@love2party.net> <200811012114.43137.max@love2party.net> <200811042342.49827.max@love2party.net> Message-ID: <87prlagayp.fsf@kobe.laptop> On Tue, 4 Nov 2008 23:42:49 +0100, Max Laier wrote: > Hi again, > > On Saturday 01 November 2008 21:14:42 I wrote: >> a thread on freebsd-stable@ [1] about problems with du(1) and compressed >> zfs filesystems got me looking for a possible solution. Attached is a diff >> for du(1) that adds two new options: >> >> -A to display the apparent size of the file instead of the used blocks. >> -B bsize to specify a custom blocksize. In particular one <512byte >> >> The GNU du(1) has --apparent-size for -A, but we don't like long options. >> That's not to say that it couldn't be added for script compat. -B is >> probably not that interesting, but it can be helpful and came for free. > > Attached is an updated patch. This refines the -B option to something more > useful (and fixes a bug in the original patch). > > From the man page: > > -B blocksize > Calculate block counts in blocksize byte blocks. This is differ- > ent from the -k, -m options or setting BLOCKSIZE and gives an > estimate of how many space the examined file hierachy would > require on a filesystem with the given blocksize. Unless in -A > mode, blocksize is rounded up to the next multiple of 512. That looks nice! With a small fix (``how _much_ space'') I like the idea a lot :) The patch fails to apply on a recent /head snapshot of du though: : keramida@kobe:/ws/bsd/src/usr.bin/du$ patch -p0 -i ~/du.AB.diff : [...] : The text leading up to this was: : -------------------------- : |Index: du.c : |=================================================================== : |--- du.c (revision 184656) : |+++ du.c (working copy) : -------------------------- : Patching file du.c using Plan A... : Hunk #1 failed at 79. : Hunk #2 failed at 99. : Hunk #3 failed at 159. : Hunk #4 succeeded at 215 (offset -2 lines). : Hunk #5 failed at 236. : Hunk #6 failed at 259. : Hunk #7 failed at 292. : Hunk #8 failed at 317. : Hunk #9 succeeded at 465 (offset -5 lines). : Hunk #10 succeeded at 482 (offset -2 lines). : 7 out of 10 hunks failed--saving rejects to du.c.rej : done : keramida@kobe:/ws/bsd/src/usr.bin/du$ Can you please refresh and repost it? From max at love2party.net Tue Nov 4 18:39:57 2008 From: max at love2party.net (Max Laier) Date: Tue Nov 4 18:40:04 2008 Subject: du -A / -B options [Re: zfs quota question] In-Reply-To: <87prlagayp.fsf@kobe.laptop> References: <200811042342.49827.max@love2party.net> <87prlagayp.fsf@kobe.laptop> Message-ID: <200811050339.52088.max@love2party.net> On Wednesday 05 November 2008 03:31:26 Giorgos Keramidas wrote: > On Tue, 4 Nov 2008 23:42:49 +0100, Max Laier wrote: > > Hi again, > > > > On Saturday 01 November 2008 21:14:42 I wrote: > >> a thread on freebsd-stable@ [1] about problems with du(1) and compressed > >> zfs filesystems got me looking for a possible solution. Attached is a > >> diff for du(1) that adds two new options: > >> > >> -A to display the apparent size of the file instead of the used blocks. > >> -B bsize to specify a custom blocksize. In particular one <512byte > >> > >> The GNU du(1) has --apparent-size for -A, but we don't like long > >> options. That's not to say that it couldn't be added for script compat. > >> -B is probably not that interesting, but it can be helpful and came for > >> free. > > > > Attached is an updated patch. This refines the -B option to something > > more useful (and fixes a bug in the original patch). > > > > From the man page: > > > > -B blocksize > > Calculate block counts in blocksize byte blocks. This is > > differ- ent from the -k, -m options or setting BLOCKSIZE and gives an > > estimate of how many space the examined file hierachy would require on a > > filesystem with the given blocksize. Unless in -A mode, blocksize is > > rounded up to the next multiple of 512. > > That looks nice! With a small fix (``how _much_ space'') I like the > idea a lot :) > > The patch fails to apply on a recent /head snapshot of du though: ... > Can you please refresh and repost it? Hum ... are you sure your snapshot is up to date? I did some style(9) cleanup in r184654/6 that you might not have yet. Anyways ... here is a diff with those revisions included. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News -------------- next part -------------- Index: du.c =================================================================== --- du.c (revision 176561) +++ du.c (working copy) @@ -73,20 +73,21 @@ static int linkchk(FTSENT *); static void usage(void); -void prthumanval(int64_t); -void ignoreadd(const char *); -void ignoreclean(void); -int ignorep(FTSENT *); +static void prthumanval(int64_t); +static void ignoreadd(const char *); +static void ignoreclean(void); +static int ignorep(FTSENT *); -int nodumpflag = 0; +static int nodumpflag = 0; +static int Aflag; +static long blocksize, cblocksize; int main(int argc, char *argv[]) { FTS *fts; FTSENT *p; - off_t savednumber = 0; - long blocksize; + off_t savednumber, curblocks; int ftsoptions; int listall; int depth; @@ -98,79 +99,91 @@ setlocale(LC_ALL, ""); Hflag = Lflag = Pflag = aflag = sflag = dflag = cflag = hflag = - lflag = 0; + lflag = Aflag = 0; save = argv; ftsoptions = 0; + savednumber = 0; + cblocksize = DEV_BSIZE; + blocksize = 0; depth = INT_MAX; SLIST_INIT(&ignores); - while ((ch = getopt(argc, argv, "HI:LPasd:chklmnrx")) != -1) + while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrx")) != -1) switch (ch) { - case 'H': - Hflag = 1; - break; - case 'I': - ignoreadd(optarg); - break; - case 'L': - if (Pflag) - usage(); - Lflag = 1; - break; - case 'P': - if (Lflag) - usage(); - Pflag = 1; - break; - case 'a': - aflag = 1; - break; - case 's': - sflag = 1; - break; - case 'd': - dflag = 1; - errno = 0; - depth = atoi(optarg); - if (errno == ERANGE || depth < 0) { - warnx("invalid argument to option d: %s", optarg); - usage(); - } - break; - case 'c': - cflag = 1; - break; - case 'h': - if (setenv("BLOCKSIZE", "512", 1) == -1) - warn( - "setenv: cannot set BLOCKSIZE=512"); - hflag = 1; - break; - case 'k': - hflag = 0; - if (setenv("BLOCKSIZE", "1024", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1024"); - break; - case 'l': - lflag = 1; - break; - case 'm': - hflag = 0; - if (setenv("BLOCKSIZE", "1048576", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1048576"); - break; - case 'n': - nodumpflag = 1; - break; - case 'r': /* Compatibility. */ - break; - case 'x': - ftsoptions |= FTS_XDEV; - break; - case '?': - default: + case 'A': + Aflag = 1; + break; + case 'B': + errno = 0; + cblocksize = atoi(optarg); + if (errno == ERANGE || cblocksize < 0) { + warnx("invalid argument to option B: %s", + optarg); usage(); + } + break; + case 'H': + Hflag = 1; + break; + case 'I': + ignoreadd(optarg); + break; + case 'L': + if (Pflag) + usage(); + Lflag = 1; + break; + case 'P': + if (Lflag) + usage(); + Pflag = 1; + break; + case 'a': + aflag = 1; + break; + case 's': + sflag = 1; + break; + case 'd': + dflag = 1; + errno = 0; + depth = atoi(optarg); + if (errno == ERANGE || depth < 0) { + warnx("invalid argument to option d: %s", + optarg); + usage(); + } + break; + case 'c': + cflag = 1; + break; + case 'h': + hflag = 1; + break; + case 'k': + hflag = 0; + blocksize = 1024; + break; + case 'l': + lflag = 1; + break; + case 'm': + hflag = 0; + blocksize = 1048576; + break; + case 'n': + nodumpflag = 1; + break; + case 'r': /* Compatibility. */ + break; + case 'x': + ftsoptions |= FTS_XDEV; + break; + case '?': + default: + usage(); + /* NOTREACHED */ } argc -= optind; @@ -204,6 +217,9 @@ if (Pflag) ftsoptions |= FTS_PHYSICAL; + if (!Aflag && (cblocksize % DEV_BSIZE) != 0) + cblocksize = howmany(cblocksize, DEV_BSIZE) * DEV_BSIZE; + listall = 0; if (aflag) { @@ -222,9 +238,14 @@ argv[1] = NULL; } - (void) getbsize(¬used, &blocksize); - blocksize /= 512; + if (blocksize == 0) + (void)getbsize(¬used, &blocksize); + if (!Aflag) { + cblocksize /= DEV_BSIZE; + blocksize /= DEV_BSIZE; + } + rval = 0; if ((fts = fts_open(argv, ftsoptions, NULL)) == NULL) @@ -232,57 +253,63 @@ while ((p = fts_read(fts)) != NULL) { switch (p->fts_info) { - case FTS_D: /* Ignore. */ - if (ignorep(p)) - fts_set(fts, p, FTS_SKIP); + case FTS_D: /* Ignore. */ + if (ignorep(p)) + fts_set(fts, p, FTS_SKIP); + break; + case FTS_DP: + if (ignorep(p)) break; - case FTS_DP: - if (ignorep(p)) - break; - p->fts_parent->fts_bignum += - p->fts_bignum += p->fts_statp->st_blocks; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); + p->fts_parent->fts_bignum += p->fts_bignum += + curblocks; - if (p->fts_level <= depth) { - if (hflag) { - (void) prthumanval(howmany(p->fts_bignum, blocksize)); - (void) printf("\t%s\n", p->fts_path); - } else { - (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_bignum, blocksize), - p->fts_path); - } + if (p->fts_level <= depth) { + if (hflag) { + prthumanval(p->fts_bignum); + (void)printf("\t%s\n", p->fts_path); + } else { + (void)printf("%jd\t%s\n", + howmany(p->fts_bignum * cblocksize, + blocksize), p->fts_path); } + } + break; + case FTS_DC: /* Ignore. */ + break; + case FTS_DNR: /* Warn, continue. */ + case FTS_ERR: + case FTS_NS: + warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); + rval = 1; + break; + default: + if (ignorep(p)) break; - case FTS_DC: /* Ignore. */ + + if (lflag == 0 && p->fts_statp->st_nlink > 1 && + linkchk(p)) break; - case FTS_DNR: /* Warn, continue. */ - case FTS_ERR: - case FTS_NS: - warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); - rval = 1; - break; - default: - if (ignorep(p)) - break; - if (lflag == 0 && - p->fts_statp->st_nlink > 1 && linkchk(p)) - break; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); - if (listall || p->fts_level == 0) { - if (hflag) { - (void) prthumanval(howmany(p->fts_statp->st_blocks, - blocksize)); - (void) printf("\t%s\n", p->fts_path); - } else { - (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_statp->st_blocks, blocksize), - p->fts_path); - } + if (listall || p->fts_level == 0) { + if (hflag) { + prthumanval(curblocks); + (void)printf("\t%s\n", p->fts_path); + } else { + (void)printf("%jd\t%s\n", + howmany(curblocks * cblocksize, + blocksize), p->fts_path); } + } - p->fts_parent->fts_bignum += p->fts_statp->st_blocks; + p->fts_parent->fts_bignum += curblocks; } savednumber = p->fts_parent->fts_bignum; } @@ -292,10 +319,11 @@ if (cflag) { if (hflag) { - (void) prthumanval(howmany(savednumber, blocksize)); - (void) printf("\ttotal\n"); + prthumanval(savednumber); + (void)printf("\ttotal\n"); } else { - (void) printf("%jd\ttotal\n", (intmax_t)howmany(savednumber, blocksize)); + (void)printf("%jd\ttotal\n", (intmax_t)howmany( + savednumber * cblocksize, blocksize)); } } @@ -348,7 +376,8 @@ free_list = le->next; free(le); } - new_buckets = malloc(new_size * sizeof(new_buckets[0])); + new_buckets = malloc(new_size * + sizeof(new_buckets[0])); } if (new_buckets == NULL) { @@ -436,12 +465,14 @@ return (0); } -void +static void prthumanval(int64_t bytes) { char buf[5]; - bytes *= DEV_BSIZE; + bytes *= cblocksize; + if (!Aflag) + bytes *= DEV_BSIZE; humanize_number(buf, sizeof(buf), bytes, "", HN_AUTOSCALE, HN_B | HN_NOSPACE | HN_DECIMAL); @@ -453,12 +484,13 @@ usage(void) { (void)fprintf(stderr, - "usage: du [-H | -L | -P] [-a | -s | -d depth] [-c] " - "[-l] [-h | -k | -m] [-n] [-x] [-I mask] [file ...]\n"); + "usage: du [-A] [-H | -L | -P] [-a | -s | -d depth] [-c] " + "[-l] [-h | -k | -m | -B bsize] [-n] [-x] [-I mask] " + "[file ...]\n"); exit(EX_USAGE); } -void +static void ignoreadd(const char *mask) { struct ignentry *ign; @@ -472,7 +504,7 @@ SLIST_INSERT_HEAD(&ignores, ign, next); } -void +static void ignoreclean(void) { struct ignentry *ign; @@ -485,7 +517,7 @@ } } -int +static int ignorep(FTSENT *ent) { struct ignentry *ign; Index: du.1 =================================================================== --- du.1 (revision 184666) +++ du.1 (working copy) @@ -40,11 +40,12 @@ .Nd display disk usage statistics .Sh SYNOPSIS .Nm +.Op Fl A .Op Fl H | L | P .Op Fl a | s | d Ar depth .Op Fl c .Op Fl l -.Op Fl h | k | m +.Op Fl h | k | m | B Ar blocksize .Op Fl n .Op Fl x .Op Fl I Ar mask @@ -60,6 +61,25 @@ .Pp The options are as follows: .Bl -tag -width indent +.It Fl A +Display the apparent size instead of the disk usage. +This can be helpful when operating on compressed volumes or sparse files. +.It Fl B Ar blocksize +Calculate block counts in +.Ar blocksize +byte blocks. +This is different from the +.Fl k, m +options or setting +.Ev BLOCKSIZE +and gives an estimate of how many space the examined file hierachy would +require on a filesystem with the given +.Ar blocksize . +Unless in +.Fl A +mode, +.Ar blocksize +is rounded up to the next multiple of 512. .It Fl H Symbolic links on the command line are followed, symbolic links in file hierarchies are not followed. @@ -136,14 +156,19 @@ If the environment variable .Ev BLOCKSIZE is set, and the -.Fl k -option is not specified, the block counts will be displayed in units of that -size block. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in units of +that size block. If .Ev BLOCKSIZE is not set, and the -.Fl k -option is not specified, the block counts will be displayed in 512-byte blocks. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in 512-byte +blocks. .El .Sh SEE ALSO .Xr df 1 , From keramida at ceid.upatras.gr Tue Nov 4 18:55:12 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Tue Nov 4 18:55:19 2008 Subject: du -A / -B options [Re: zfs quota question] In-Reply-To: <200811050339.52088.max@love2party.net> (Max Laier's message of "Wed, 5 Nov 2008 03:39:51 +0100") References: <200811042342.49827.max@love2party.net> <87prlagayp.fsf@kobe.laptop> <200811050339.52088.max@love2party.net> Message-ID: <87myge6fwf.fsf@kobe.laptop> On Wed, 5 Nov 2008 03:39:51 +0100, Max Laier wrote: >> That looks nice! With a small fix (``how _much_ space'') I like the >> idea a lot :) >> >> The patch fails to apply on a recent /head snapshot of du though: > ... >> Can you please refresh and repost it? > > Hum ... are you sure your snapshot is up to date? I did some style(9) > cleanup in r184654/6 that you might not have yet. Anyways ... here is > a diff with those revisions included. Ah, that's it then. I ran 'svn up' at the change right *before* the HPS-USB commit and the latest du stuff was missing. Thanks :) The patch works fine here. I only had to make a tiny change, to avoid a core dump when the -B blocksize is invalid as an integer and it caused a division by zero in fts: $ ./du -B abcde Floating point exception: 8 (core dumped) Here's a diff of all the local changes I made: %%% diff -r a3da3ebb57ab usr.bin/du/du.1 --- a/usr.bin/du/du.1 Wed Nov 05 04:47:07 2008 +0200 +++ b/usr.bin/du/du.1 Wed Nov 05 04:53:28 2008 +0200 @@ -72,7 +72,7 @@ .Fl k, m options or setting .Ev BLOCKSIZE -and gives an estimate of how many space the examined file hierachy would +and gives an estimate of how much space the examined file hierachy would require on a filesystem with the given .Ar blocksize . Unless in @@ -160,7 +160,7 @@ or .Fl h options are not specified, the block counts will be displayed in units of -that size block. +that block size. If .Ev BLOCKSIZE is not set, and the diff -r a3da3ebb57ab usr.bin/du/du.c --- a/usr.bin/du/du.c Wed Nov 05 04:47:07 2008 +0200 +++ b/usr.bin/du/du.c Wed Nov 05 04:53:28 2008 +0200 @@ -117,7 +117,7 @@ case 'B': errno = 0; cblocksize = atoi(optarg); - if (errno == ERANGE || cblocksize < 0) { + if (errno == ERANGE || cblocksize <= 0) { warnx("invalid argument to option B: %s", optarg); usage(); %%% From steve at localhost.lu Wed Nov 5 04:50:44 2008 From: steve at localhost.lu (Steve Clement) Date: Wed Nov 5 04:50:52 2008 Subject: Compile issues on 7.0-p5 Message-ID: <491192CD.4050306@localhost.lu> Hi List, I have the following altq issue: /usr/src/sys/netinet/tcp_usrreq.c: In function 'tcp_fill_info': /usr/src/sys/netinet/tcp_usrreq.c:1232: error: 'struct tcp_info' has no member named 'tcpi_rcv_nxt' it seems that tcp.h has either been updated or something else has changed. the following altq options are in my kernel: # ALTQ Support options ALTQ options ALTQ_CBQ options ALTQ_RED options ALTQ_RIO options ALTQ_HFSC options ALTQ_CDNR options ALTQ_PRIQ Cheers for help, Steve cc -c -O2 -pipe -fno-strict-aliasing -march=pentium4 -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -Werror /usr/src/sys/netinet/tcp_timer.c cc -c -O2 -pipe -fno-strict-aliasing -march=pentium4 -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -Werror /usr/src/sys/netinet/tcp_timewait.c cc -c -O2 -pipe -fno-strict-aliasing -march=pentium4 -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -Werror /usr/src/sys/netinet/tcp_usrreq.c /usr/src/sys/netinet/tcp_usrreq.c: In function 'tcp_fill_info': /usr/src/sys/netinet/tcp_usrreq.c:1232: error: 'struct tcp_info' has no member named 'tcpi_rcv_nxt' /usr/src/sys/netinet/tcp_usrreq.c:1235: error: 'struct tcp_info' has no member named 'tcpi_snd_nxt' /usr/src/sys/netinet/tcp_usrreq.c:1239: error: 'TCPI_OPT_TOE' undeclared (first use in this function) /usr/src/sys/netinet/tcp_usrreq.c:1239: error: (Each undeclared identifier is reported only once /usr/src/sys/netinet/tcp_usrreq.c:1239: error: for each function it appears in.) *** Error code 1 Stop in /usr/obj/usr/src/sys/IPF. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. From rihad at mail.ru Wed Nov 5 08:30:37 2008 From: rihad at mail.ru (rihad) Date: Wed Nov 5 08:46:57 2008 Subject: Asynchronous pipe I/O Message-ID: <4911A23B.7050104@mail.ru> Imagine this shell pipeline: sh prog1 | sh prog2 As given above, prog1 blocks if prog2 hasn't yet read previously written data (actually, newline separated commands) or is busy. What I want is for prog1 to never block: sh prog1 | buffer | sh prog2 I first thought that the aptly named misc/buffer port would do exactly what I wanted: buffering prog1 output for prog2 to read it at its earliest convenience. That way prog1 would never block (unless it hit buffer's memory limits). Alas, misc/buffer was originally designed for tape backups, and despite its author's stating its applicability to other uses: "This is a program designed initially to speed up writing tapes on remote tape drives, but may be used as a general pipe buffering utility." buffer never starts writing unless the limit given by -s is crossed, which is 10 kbytes by default and cannot be less than 496 bytes, which is too much for me. Ideally I want it to start writing immediately, whenever new data hits its pools. Unfortunately, the -p 0 option doesn't work either: -s size Size in bytes of each block. The default blocksize is 10k to match the normal output of the tar(1) program. -p percentage Only start a write when the given percentage of the internal queue is full. A percentage around 75 often proves best. Defaults to zero. Wouldn't such an intermediary tool be a great way to boost performance for certain types of solutions? Thanks for any tips (Sorry if this was an inappropriate place to ask) From mezz7 at cox.net Wed Nov 5 09:08:04 2008 From: mezz7 at cox.net (Jeremy Messenger) Date: Wed Nov 5 09:08:11 2008 Subject: preallocate files without (posix_)fallocate or fcntl's F_PREALLOCATE? Message-ID: Hello folks, At first, I don't really know C that well so... It's more of request or suggest if anyone is bored or interest to port on FreeBSD. :-) Recently, Transmission has added preallocate files to prevent disk fragmentation but FreeBSD does not has any of (posix_)fallocate or fcntl's F_PREALLOCATE (checked in RELENG_7's src/* and manpages). Here's what it looks like: ---------------------------------------- static int preallocateFile( int fd UNUSED, uint64_t length UNUSED ) { #ifdef HAVE_FALLOCATE return fallocate( fd, 0, offset, length ); #elif defined(HAVE_POSIX_FALLOCATE) return posix_fallocate( fd, 0, length ); #elif defined(SYS_DARWIN) fstore_t fst; fst.fst_flags = F_ALLOCATECONTIG; fst.fst_posmode = F_PEOFPOSMODE; fst.fst_offset = 0; fst.fst_length = length; fst.fst_bytesalloc = 0; return fcntl( fd, F_PREALLOCATE, &fst ); #else #warning no known method to preallocate files on this platform return -1; #endif } ---------------------------------------- You can see a bit more over at http://trac.transmissionbt.com/changeset/7051 .. You can also point me to some sources if it exists for I can try do it by myself, but with lacking C knowledge is a bit hard for me at the most of time. I don't know if UFS does not need or does need it. If anyone is interesting to write simple one for FreeBSD will be cool. Thanks! Cheers, Mezz -- mezz7@cox.net - mezz@FreeBSD.org FreeBSD GNOME Team http://www.FreeBSD.org/gnome/ - gnome@FreeBSD.org From neldredge at math.ucsd.edu Wed Nov 5 09:10:27 2008 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Wed Nov 5 09:10:34 2008 Subject: Asynchronous pipe I/O In-Reply-To: <4911A23B.7050104@mail.ru> References: <4911A23B.7050104@mail.ru> Message-ID: On Wed, 5 Nov 2008, rihad wrote: > Imagine this shell pipeline: > > sh prog1 | sh prog2 > > > As given above, prog1 blocks if prog2 hasn't yet read previously written > data (actually, newline separated commands) or is busy. What I want is > for prog1 to never block: > > sh prog1 | buffer | sh prog2 [and misc/buffer is unsuitable] I found an old piece of code laying around that I wrote for this purpose. Looking at it, I can see a number of inefficiencies, but it might do in a pinch. You're welcome to use it; I hereby release it to the public domain. Another hack that you could use, if you don't mind storing the buffer on disk rather than memory, is sh prog1 > tmpfile & tail -f -c +0 tmpfile | sh prog2 Here's my program. /* Buffering filter. */ #include #include #include #include #include #include /* Size of a single buffer. */ #define BUFSIZE 512 struct buffer { struct buffer *next; size_t length; unsigned char buf[BUFSIZE]; }; struct buffer *reader; struct buffer *writer; int max_mem = 100 * 1024; int current_mem; #define OK 0 #define WAIT 1 #define GIVEUP 2 int read_one (int fd) { int result; if (current_mem > (max_mem - sizeof(*reader->next))) { fprintf(stderr, "Reached max_mem!\n"); return WAIT; } /* Get a new buffer. */ reader->next = malloc(sizeof(*reader->next)); if (reader->next) { current_mem += sizeof(*reader->next); fprintf(stderr, "\rReading: \t%u bytes in buffer ", current_mem); } else { fprintf(stderr, "Virtual memory exhausted\n"); return WAIT; } reader = reader->next; reader->next = NULL; result = read(fd, reader->buf, BUFSIZE); if (result > 0) reader->length = result; else if (result == 0) { fprintf(stderr, "Hit EOF on reader\n"); return GIVEUP; } else if (result < 0) { fprintf(stderr, "Error on reader: %s\n", strerror(errno)); return GIVEUP; } return OK; } int write_one (int fd) { struct buffer *newwriter; if (reader == writer) return WAIT; /* the reader owns the last buffer */ if (writer->length > 0) { int result; result = write(fd, writer->buf, writer->length); if (result == 0) { fprintf(stderr, "Hit EOF on writer\n"); return GIVEUP; } else if (result < 0) { fprintf(stderr, "Error on writer: %s\n", strerror(errno)); return GIVEUP; } } newwriter = writer->next; free(writer); current_mem -= sizeof(*writer); fprintf(stderr, "\rWriting: \t%u bytes in buffer ", current_mem); writer = newwriter; return OK; } void move_data(int in_fd, int out_fd) { int reader_state = OK; int writer_state = OK; int maxfd = ((in_fd > out_fd) ? in_fd : out_fd) + 1; reader = malloc(sizeof(*reader)); if (!reader) { fprintf(stderr, "No memory at all!\n"); return; } reader->next = NULL; reader->length = 0; writer = reader; current_mem = sizeof(*reader); while (1) /* break when done */ { int result; fd_set read_set, write_set; FD_ZERO(&read_set); FD_ZERO(&write_set); if (reader_state == OK) FD_SET(in_fd, &read_set); if (writer_state == OK) FD_SET(out_fd, &write_set); result = select(maxfd, &read_set, &write_set, NULL, NULL); /* If we're ready to do something, do it. Also let the other end get a chance if something changed. */ if (FD_ISSET(in_fd, &read_set)) { reader_state = read_one(in_fd); if (writer_state == WAIT) writer_state = OK; } if (FD_ISSET(out_fd, &write_set)) { writer_state = write_one(out_fd); if (reader_state == WAIT) reader_state = OK; } /* Check for termination */ if (writer_state == GIVEUP) break; /* can't write any more */ if (reader_state == GIVEUP && writer_state == WAIT) break; /* can't read any more, and wrote all we have */ } } int main(void) { move_data(0, 1); return 0; } -- Nate Eldredge neldredge@math.ucsd.edu From peterjeremy at optushome.com.au Wed Nov 5 10:13:13 2008 From: peterjeremy at optushome.com.au (Peter Jeremy) Date: Wed Nov 5 10:13:19 2008 Subject: Asynchronous pipe I/O In-Reply-To: <4911A23B.7050104@mail.ru> References: <4911A23B.7050104@mail.ru> Message-ID: <20081105181309.GB51239@server.vk2pj.dyndns.org> On 2008-Nov-05 17:40:11 +0400, rihad wrote: >Imagine this shell pipeline: > >sh prog1 | sh prog2 > > >As given above, prog1 blocks if prog2 hasn't yet read previously written >data (actually, newline separated commands) or is busy. What I want is >for prog1 to never block: > >sh prog1 | buffer | sh prog2 There's also misc/mbuffer which is supposed to be an enhancement of misc/buffer - though I haven't used either. I have a program I wrote to do this but it's not in a releasable state. >Wouldn't such an intermediary tool be a great way to boost performance >for certain types of solutions? I've found that for dump|restore or dump|gzip, I can get quite significant speedups by adding a buffer that is several hundred MB in the middle. -- Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081105/6de97308/attachment.pgp From ed at 80386.nl Wed Nov 5 10:28:05 2008 From: ed at 80386.nl (Ed Schouten) Date: Wed Nov 5 10:28:12 2008 Subject: preallocate files without (posix_)fallocate or fcntl's F_PREALLOCATE? In-Reply-To: References: Message-ID: <20081105182803.GG1165@hoeg.nl> Hello, Not entirely related to this topic, but looking at the standards, it seems we don't implement posix_fadvise() either. It seems we could implement this as a no-op for now. It has no real advantage if we do, but say, one day we gain real posix_fadvise() support, we already have a bunch of binaries that have these performance optimisations compiled in. Any comments? -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081105/3ea43377/attachment.pgp From rihad at mail.ru Wed Nov 5 11:10:40 2008 From: rihad at mail.ru (rihad) Date: Wed Nov 5 11:16:59 2008 Subject: Asynchronous pipe I/O In-Reply-To: References: <4911A23B.7050104@mail.ru> Message-ID: <4911EFAC.7090909@mail.ru> Nate Eldredge wrote: > On Wed, 5 Nov 2008, rihad wrote: > >> Imagine this shell pipeline: >> >> sh prog1 | sh prog2 >> >> >> As given above, prog1 blocks if prog2 hasn't yet read previously written >> data (actually, newline separated commands) or is busy. What I want is >> for prog1 to never block: >> >> sh prog1 | buffer | sh prog2 > > [and misc/buffer is unsuitable] > > I found an old piece of code laying around that I wrote for this > purpose. Looking at it, I can see a number of inefficiencies, but it > might do in a pinch. You're welcome to use it; I hereby release it to > the public domain. > > Another hack that you could use, if you don't mind storing the buffer on > disk rather than memory, is > > sh prog1 > tmpfile & > tail -f -c +0 tmpfile | sh prog2 > Thanks, but I was looking for how to get some port do that useful functionality for me. Strange as it is, there doesn't seem to be such a tool handy, despite the "small tool doing one thing good" philosophy of Unix! It's invaluable for optimizing certain tasks and isn't coupled to anything: as generic as could be. Perhaps I should also email buffer's author for his opinion, and ask him why -p 0 (default) doesn't do the Right Thing. > Here's my program. Thanks again, I'm sure I'll use your code if I can't get the port version to work. From rihad at mail.ru Wed Nov 5 11:21:18 2008 From: rihad at mail.ru (rihad) Date: Wed Nov 5 13:22:24 2008 Subject: Asynchronous pipe I/O In-Reply-To: <20081105181309.GB51239@server.vk2pj.dyndns.org> References: <4911A23B.7050104@mail.ru> <20081105181309.GB51239@server.vk2pj.dyndns.org> Message-ID: <4911F229.2010502@mail.ru> Peter Jeremy wrote: > On 2008-Nov-05 17:40:11 +0400, rihad wrote: >> Imagine this shell pipeline: >> >> sh prog1 | sh prog2 >> >> >> As given above, prog1 blocks if prog2 hasn't yet read previously written >> data (actually, newline separated commands) or is busy. What I want is >> for prog1 to never block: >> >> sh prog1 | buffer | sh prog2 > > There's also misc/mbuffer which is supposed to be an enhancement of > misc/buffer - though I haven't used either. I have a program I wrote > to do this but it's not in a releasable state. > That thing only works for certain types of devices (tape?): $ while :; do echo hi; sleep 1; done | mbuffer -q | cat warning: Could not stat output device (unsupported by system)! This can result in incorrect written data when using multiple volumes. Continue at your own risk! ^C $ While writing this email I tried another incantation, and buffer finally worked! $ while :; do echo hi; sleep 1; done | buffer -s1 -b100 | cat hi hi hi hi hi ^C (each line was output one second apart). Once again thank to everybody participating! >> Wouldn't such an intermediary tool be a great way to boost performance >> for certain types of solutions? > > I've found that for dump|restore or dump|gzip, I can get quite significant > speedups by adding a buffer that is several hundred MB in the middle. > Well, if OS buffers aren't good enough, then throwing some memory at disk I/O surely helps ;) From kientzle at freebsd.org Wed Nov 5 13:37:15 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Wed Nov 5 13:37:21 2008 Subject: converting strings from utf8 In-Reply-To: References: Message-ID: <49121206.9090804@freebsd.org> Maksim Yevmenkin wrote: > > can i use wcstombs(3) to convert a string presented in utf8 into > current locale? basically i'm looking for something like iconv from > ports but included into base system. This isn't as easy as it should be, unfortunately. First, UTF-8 is itself a multibyte encoding, so you have to first convert to wide characters before you can use wcstombs(). You could in theory use the following: * Set locale to UTF-8 * use mbstowcs() to convert UTF-8 into wide characters * Set locale to your preferred locale * use wcstombs() to convert wide characters to your locale Besides being ugly, the locale names themselves are not standardized, so it's hard to do this portably. For a lot of applications, the error handling in wcstombs() is also troublesome; it rejects the entire string if any one character can't be converted. When I had to do this for libarchive, where the code had to be very portable (which precluded using iconv), I ended up doing the following: * Wrote my own converter from UTF-8 to wide characters (fortunately, UTF-8 is pretty simple to decode; this is about 20-30 lines of C) * Used wctomb() to convert one character at a time from wide characters to the current locale. I've found that wctomb() is more portable than a lot of the other functions (I think it's in C89, whereas a lot of the other standard conversion routines were introduced in C99) and provides better error-handling capabilities since it operates on one character at a time (so you can, for instance, convert characters that aren't supported in the current locale into '?' or some kind of \-escape). Feel free to copy any of my code from libarchive if it helps. Tim From jhb at freebsd.org Wed Nov 5 14:51:23 2008 From: jhb at freebsd.org (John Baldwin) Date: Wed Nov 5 14:51:30 2008 Subject: preallocate files without (posix_)fallocate or fcntl's F_PREALLOCATE? In-Reply-To: References: Message-ID: <200811051717.34933.jhb@freebsd.org> On Wednesday 05 November 2008 12:08:11 pm Jeremy Messenger wrote: > Hello folks, > > At first, I don't really know C that well so... It's more of request or > suggest if anyone is bored or interest to port on FreeBSD. :-) > > Recently, Transmission has added preallocate files to prevent disk > fragmentation but FreeBSD does not has any of (posix_)fallocate or fcntl's > F_PREALLOCATE (checked in RELENG_7's src/* and manpages). Here's what it > looks like: > > ---------------------------------------- > static int > preallocateFile( int fd UNUSED, uint64_t length UNUSED ) > { > #ifdef HAVE_FALLOCATE > > return fallocate( fd, 0, offset, length ); > > #elif defined(HAVE_POSIX_FALLOCATE) > > return posix_fallocate( fd, 0, length ); > > #elif defined(SYS_DARWIN) > > fstore_t fst; > fst.fst_flags = F_ALLOCATECONTIG; > fst.fst_posmode = F_PEOFPOSMODE; > fst.fst_offset = 0; > fst.fst_length = length; > fst.fst_bytesalloc = 0; > return fcntl( fd, F_PREALLOCATE, &fst ); > > #else > > #warning no known method to preallocate files on this platform > return -1; > > #endif > } > ---------------------------------------- > > You can see a bit more over at > http://trac.transmissionbt.com/changeset/7051 .. You can also point me to > some sources if it exists for I can try do it by myself, but with lacking > C knowledge is a bit hard for me at the most of time. I don't know if UFS > does not need or does need it. If anyone is interesting to write simple > one for FreeBSD will be cool. Thanks! UFS would benefit. See the notes on MAP_NOSYNC in mmap(2). Peter Wemm (peter@) probably can give a more detailed suggestion of what you would need to do. Doing it efficiently would probably require a new VOP_ALLOCATE() or some such. -- John Baldwin From rihad at mail.ru Wed Nov 5 23:52:53 2008 From: rihad at mail.ru (rihad) Date: Thu Nov 6 04:12:46 2008 Subject: Asynchronous pipe I/O In-Reply-To: References: <4911A23B.7050104@mail.ru> Message-ID: <4912A251.8010604@mail.ru> Nate Eldredge wrote: > sh prog1 > tmpfile & > tail -f -c +0 tmpfile | sh prog2 > BTW, I don't think this would solve my problem as tail -f would block waiting for more data, as would prog2, so after prog1 finishes writing data they would block indefinitely on input. I forgot to mention this was supposed to be a short-lived pipeline, not daemons, as you had probably thought. Also, I've already learned how to use misc/buffer correctly so the problem's gone. Thank you for your time. From petefrench at ticketswitch.com Thu Nov 6 04:58:02 2008 From: petefrench at ticketswitch.com (Pete French) Date: Thu Nov 6 05:13:17 2008 Subject: du -A / -B options [Re: zfs quota question] In-Reply-To: <200811012114.43137.max@love2party.net> Message-ID: > Any objections against the general concept? It's rather complicated to get > the apparent size of a directory hierarchy without scripting. I often wonder > if some hierarchy will fit on a CD/DVD and compressed zfs makes this really > difficult. It's a definite thumbs up from me - this is exactly what I was looking for in the original thread. Very nice couple of new options. -pete. From keramida at ceid.upatras.gr Thu Nov 6 19:34:58 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Thu Nov 6 19:35:14 2008 Subject: includes, configure, /usr/lib vs. /usr/local/lib, and linux coders In-Reply-To: <539c60b90811031412y35e2599cneca5912ada43b081@mail.gmail.com> (Steve Franks's message of "Mon, 3 Nov 2008 15:12:34 -0700") References: <539c60b90810311123w2aa94b8akcd0a5d0fe791885a@mail.gmail.com> <539c60b90810311230i11460966la7ff35b0093642ec@mail.gmail.com> <87r65vl9ur.fsf@kobe.laptop> <539c60b90811031412y35e2599cneca5912ada43b081@mail.gmail.com> Message-ID: <87tzakqkdn.fsf@kobe.laptop> On Mon, 3 Nov 2008 15:12:34 -0700, "Steve Franks" wrote: > That's alot of good info. It should go in the porter's handbook, > maybe... Hi Steve, Probably not. What I wrote is specific to the GNU build system. We have many ports that use configure scripts and makefiles generated from various versions of the GNU build tools, but ports are ``different''. We are not the _maintainers_ of the main source code of all the ported applications. We just have to make sure they build on FreeBSD, and that's pretty much all of it. For example, if the source tree of a port includes a `configure.in' that is broken on FreeBSD and Cygwin, we don't really have to ``fix'' both of these. If it builds correctly on FreeBSD, we are done. This may not be enough for Cygwin users, but we are not out to fix everyone's code to build on everybody else's system. That would be an insane amount of work for a very doubtful amount of gain :) > So, if I'm looking to make a port, which one of those people should I > be acting as? Maintainer? That's FreeBSD-port-terminology you are > using, correct? FreeBSD Porters are a separate category. They usually fall in the category of `builder' I mentioned in the original post, but they have to provide the tools for `packagers' too, in the form of Makefiles, scripts and packaging lists that allow others to configure, build and package the ``vendor'' code for some FreeBSD version. When I mentioned `maintainer', `builder' and `packager' roles in the original post I didn't mean *FreeBSD-maintainer* but the actual person or team that maintains the upstream source of a program. HTH, Giorgos From sem at FreeBSD.org Fri Nov 7 11:27:49 2008 From: sem at FreeBSD.org (Sergey Matveychuk) Date: Fri Nov 7 11:27:56 2008 Subject: LIST_REMOVE problem Message-ID: <491491E5.8090805@FreeBSD.org> Hi! I wonder how this example from queue(3) works: while (!LIST_EMPTY(&head)) { /* List Deletion. */ n1 = LIST_FIRST(&head); LIST_REMOVE(n1, entries); free(n1); } If we'll take a look at code, LIST_REMOVE don't change a head pointer if you remove the first element: #define LIST_FIRST(head) ((head)->lh_first) ... #define LIST_REMOVE(elm, field) do { \ if (LIST_NEXT((elm), field) != NULL) \ LIST_NEXT((elm), field)->field.le_prev = \ (elm)->field.le_prev; \ *(elm)->field.le_prev = LIST_NEXT((elm), field); \ } while (0) So n1 in the example above will point to removed (and freed) element. -- Dixi. Sem. From joerg at britannica.bec.de Fri Nov 7 11:29:54 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Fri Nov 7 11:30:02 2008 Subject: LIST_REMOVE problem In-Reply-To: <491491E5.8090805@FreeBSD.org> References: <491491E5.8090805@FreeBSD.org> Message-ID: <20081107192520.GA17933@britannica.bec.de> On Fri, Nov 07, 2008 at 10:07:17PM +0300, Sergey Matveychuk wrote: > If we'll take a look at code, LIST_REMOVE don't change a head pointer if > you remove the first element: It does via le_priv. Joerg From sem at FreeBSD.org Sat Nov 8 03:07:59 2008 From: sem at FreeBSD.org (Sergey Matveychuk) Date: Sat Nov 8 03:08:30 2008 Subject: LIST_REMOVE problem In-Reply-To: <20081107192520.GA17933@britannica.bec.de> References: <491491E5.8090805@FreeBSD.org> <20081107192520.GA17933@britannica.bec.de> Message-ID: <4915730C.3050303@FreeBSD.org> Joerg Sonnenberger wrote: > On Fri, Nov 07, 2008 at 10:07:17PM +0300, Sergey Matveychuk wrote: >> If we'll take a look at code, LIST_REMOVE don't change a head pointer if >> you remove the first element: > > It does via le_priv. OK, thanks. I see now. I'll try to find out why my code don't work this way on 6.4. -- Dixi. Sem. From eitanadlerlist at gmail.com Sun Nov 9 14:58:30 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sun Nov 9 14:58:36 2008 Subject: uname -c: alias for uname -rms Message-ID: <491764C3.5020504@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Personally I find that uname -rms to be the most useful set of uname flags (especially when providing support in a public setting). inspiration: http://groups.google.com/group/lucky.freebsd.chat.digest/browse_frm/thread/b0c44dad9f63f2bd Any comments on the following patch: - --- uname.c 2008-11-09 17:26:52.000000000 -0500 +++ uname.old 2008-11-09 17:26:41.000000000 -0500 @@ -88,11 +88,14 @@ setup_get(); flags = 0; - - while ((ch = getopt(argc, argv, "aimnprsv")) != -1) + while ((ch = getopt(argc, argv, "acimnprsv")) != -1) switch(ch) { case 'a': flags |= (MFLAG | NFLAG | RFLAG | SFLAG | VFLAG); break; + case 'c': + flags |= (RFLAG | MFLAG | SFLAG); + break; case 'i': flags |= IFLAG; break; - -- Eitan Adler -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkXZMMACgkQtl8kq+nCzNFtJQCfcabTV0uAAQhRhgYcSaSqM2Zt H2EAoItxfCpFSGVmcziEiI0uCMz0RGo5 =PFEl -----END PGP SIGNATURE----- From kientzle at freebsd.org Sun Nov 9 16:39:31 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Nov 9 16:39:37 2008 Subject: uname -c: alias for uname -rms In-Reply-To: <491764C3.5020504@gmail.com> References: <491764C3.5020504@gmail.com> Message-ID: <491782BD.9090009@freebsd.org> Are there any precedents for this option? If not, then I don't see the point. If you have to tell someone to send you the output of uname, you may as well tell them to use uname -rms. (I prefer uname -v personally.) Tim Eitan Adler wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Personally I find that uname -rms to be the most useful set of uname > flags (especially when providing support in a public setting). > > inspiration: > http://groups.google.com/group/lucky.freebsd.chat.digest/browse_frm/thread/b0c44dad9f63f2bd > > Any comments on the following patch: > > - --- uname.c 2008-11-09 17:26:52.000000000 -0500 > +++ uname.old 2008-11-09 17:26:41.000000000 -0500 > @@ -88,11 +88,14 @@ > setup_get(); > flags = 0; > > - - while ((ch = getopt(argc, argv, "aimnprsv")) != -1) > + while ((ch = getopt(argc, argv, "acimnprsv")) != -1) > switch(ch) { > case 'a': > flags |= (MFLAG | NFLAG | RFLAG | SFLAG | VFLAG); > break; > + case 'c': > + flags |= (RFLAG | MFLAG | SFLAG); > + break; > case 'i': > flags |= IFLAG; > break; > > - -- > Eitan Adler > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.9 (FreeBSD) > > iEYEARECAAYFAkkXZMMACgkQtl8kq+nCzNFtJQCfcabTV0uAAQhRhgYcSaSqM2Zt > H2EAoItxfCpFSGVmcziEiI0uCMz0RGo5 > =PFEl > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From eprod at rootshell.be Sun Nov 9 17:12:57 2008 From: eprod at rootshell.be (Eugene Prodeguene) Date: Sun Nov 9 17:13:04 2008 Subject: uname -c: alias for uname -rms In-Reply-To: <491782BD.9090009@freebsd.org> References: <491764C3.5020504@gmail.com> <491782BD.9090009@freebsd.org> Message-ID: Not to mention Richard Stallman would be non to happy to see the -rms option set replaced with -c. j/k. On Sun, 9 Nov 2008, Tim Kientzle wrote: > Are there any precedents for this option? > > If not, then I don't see the point. If you have to tell someone to send you > the output of uname, you may as well tell them to use uname -rms. > (I prefer uname -v personally.) > > Tim > > > Eitan Adler wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Personally I find that uname -rms to be the most useful set of uname >> flags (especially when providing support in a public setting). >> >> inspiration: >> http://groups.google.com/group/lucky.freebsd.chat.digest/browse_frm/thread/b0c44dad9f63f2bd >> >> Any comments on the following patch: >> >> - --- uname.c 2008-11-09 17:26:52.000000000 -0500 >> +++ uname.old 2008-11-09 17:26:41.000000000 -0500 >> @@ -88,11 +88,14 @@ >> setup_get(); >> flags = 0; >> >> - - while ((ch = getopt(argc, argv, "aimnprsv")) != -1) >> + while ((ch = getopt(argc, argv, "acimnprsv")) != -1) >> switch(ch) { >> case 'a': >> flags |= (MFLAG | NFLAG | RFLAG | SFLAG | VFLAG); >> break; >> + case 'c': >> + flags |= (RFLAG | MFLAG | SFLAG); >> + break; >> case 'i': >> flags |= IFLAG; >> break; >> >> - -- >> Eitan Adler >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2.0.9 (FreeBSD) >> >> iEYEARECAAYFAkkXZMMACgkQtl8kq+nCzNFtJQCfcabTV0uAAQhRhgYcSaSqM2Zt >> H2EAoItxfCpFSGVmcziEiI0uCMz0RGo5 >> =PFEl >> -----END PGP SIGNATURE----- >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> >> > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From shilp.kamal at yahoo.com Sun Nov 9 18:26:19 2008 From: shilp.kamal at yahoo.com (Kamlesh Patel) Date: Sun Nov 9 18:31:16 2008 Subject: What happen when FreeBSD boot first time???????? Message-ID: <864116.63596.qm@web45402.mail.sp1.yahoo.com> Hi Friends, I am working on FreeBSD Virtual Memory improvement algorithm. How VM allocate pages first time when system boot? How phys_avail[] have values in sys/vm/vm_Page.c line 228? Thanks Kamlesh From eitanadlerlist at gmail.com Sun Nov 9 18:38:25 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sun Nov 9 18:38:31 2008 Subject: uname -c: alias for uname -rms In-Reply-To: <491782BD.9090009@freebsd.org> References: <491764C3.5020504@gmail.com> <491782BD.9090009@freebsd.org> Message-ID: <49179E9A.2020507@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Kientzle wrote: > Are there any precedents for this option? Not really - its more like the recent cp -a switch: an alias for a common set of switches. > > If not, then I don't see the point. If you have to tell someone to send > you the output of uname, you may as well tell them to use uname -rms. > (I prefer uname -v personally.) Some people don't like giving out hostnames or kernel names for some strange reason that I can't figure out - -- Eitan Adler -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkXnpoACgkQtl8kq+nCzNGnUQCfXX2jEExBmX0gpqyvuegAmHei qXAAnRDWcJhwJu3oPAUYLp1Ydghg/onQ =2TSg -----END PGP SIGNATURE----- From gonzo at bluezbox.com Sun Nov 9 19:20:21 2008 From: gonzo at bluezbox.com (Oleksandr Tymoshenko) Date: Sun Nov 9 19:20:28 2008 Subject: What happen when FreeBSD boot first time???????? In-Reply-To: <864116.63596.qm@web45402.mail.sp1.yahoo.com> References: <864116.63596.qm@web45402.mail.sp1.yahoo.com> Message-ID: <49179F7C.7020200@bluezbox.com> Kamlesh Patel wrote: > Hi Friends, > > I am working on FreeBSD Virtual Memory improvement algorithm. > How VM allocate pages first time when system boot? > How phys_avail[] have values in sys/vm/vm_Page.c line 228? It's set by machine-dependent part of code. grep for phys_avail[] in i386/amd64/arm/etc.. From shilp.kamal at yahoo.com Sun Nov 9 19:27:14 2008 From: shilp.kamal at yahoo.com (Kamlesh Patel) Date: Sun Nov 9 22:19:27 2008 Subject: What happen when FreeBSD boot first time???????? Message-ID: <862250.48654.qm@web45405.mail.sp1.yahoo.com> Hi Friends, I am working on FreeBSD Virtual Memory improvement algorithm. How VM allocate pages first time when system boot? How phys_avail[] have values in sys/vm/vm_Page.c line 228? Thanks Kamlesh It's set by machine-dependent part of code. grep for phys_avail[] in i386/amd64/arm/etc.. Oleksandr Tymoshenko" From: Oleksandr Tymoshenko > Subject: Re: What happen when FreeBSD boot first time???????? > To: shilp.kamal@yahoo.com > Cc: freebsd-hackers@freebsd.org, mayur.shardul@gmail.com > Date: Sunday, November 9, 2008, 6:42 PM > Kamlesh Patel wrote: > > Hi Friends, > > > > I am working on FreeBSD Virtual Memory improvement > algorithm. > > How VM allocate pages first time when system boot? > > How phys_avail[] have values in sys/vm/vm_Page.c line > 228? > It's set by machine-dependent part of code. grep > for > phys_avail[] in i386/amd64/arm/etc.. > ________________________________________ From ravi.murty at intel.com Mon Nov 10 09:58:09 2008 From: ravi.murty at intel.com (Murty, Ravi) Date: Mon Nov 10 09:58:16 2008 Subject: Typo in ULE in FreeBSD 8.0 -- that's not really a bug Message-ID: <6D5D25EA3941074EB7734E51B16687040AD02A77@orsmsx506.amr.corp.intel.com> Hello All, I have been playing with ULE in 8.0 and while staring at tdq_notify noticed an interesting (and what seems like a typo) problem. The intention of the function is obvious, send an IPI to notify the remote CPU of some new piece of work. In the case where there is no IPI currently pending on the target CPU and this thread should be preempting what's running there, the code checks in td (passed in as a parameter) is the IDLE thread (TDF_IDLETD). If so, it checks the state and sees if idle is RUNNING and if so figures it will notice this new work and we don't really need to send an expensive IPI. However, why would td (parameter) ever be the IDLE thread? It almost seems like this check will always fail and we end up sending a hard IPI to the target CPU which works, but may not be needed. May be we wanted to use PCPU->curthread instead of td? Thanks Ravi From avg at icyb.net.ua Tue Nov 11 05:18:26 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Tue Nov 11 05:18:32 2008 Subject: usb keyboard dying at loader prompt In-Reply-To: <4912E462.4090608@icyb.net.ua> References: <4912E462.4090608@icyb.net.ua> Message-ID: <491981D0.7060100@icyb.net.ua> on 06/11/2008 14:34 Andriy Gapon said the following: > I have a quite strange problem. > This is with 7-BETA amd64. > All of USB is out of kernel and is loaded via modules. > BIOS has "Legacy USB" enabled. > I have only a USB keyboard, no PS/2 port. > > The keyboard works file in BIOS and for selecting boot device in boot0 > menu. It also works in loader menu. If in the menu I select to go to > loader prompt then it works for about 5 seconds and then "dies" - no > reaction to key presses, no led change, nothing. > I haven't actually verified if the keyboard would still work if I stayed > in loader menu for longer than ~10 seconds. > > This doesn't happen if USB is built into kernel. > > Weird... I did more experimentation and the behavior seems to be quite random - sometimes keyboard works ok for long time in all places, sometimes it stops working after some period of time, sometimes it doesn't work from the start and couple of times I experienced boot process going astray. Not sure what stage that was, there were endless messages spewed on the screen very fast, I couldn't read them. This leads me to the following "crazy" question - is it possible that our boot chain corrupts some vital BIOS memory? I think loader would be a primary suspect. I am not sure of anything, but a wild guess is that RAM where BIOS stores some USB-related stuff gets corrupted. Maybe it's overwritten when kernel and modules are loaded... -- Andriy Gapon From kuku at kukulies.org Tue Nov 11 10:20:58 2008 From: kuku at kukulies.org (Christoph Kukulies) Date: Tue Nov 11 10:21:04 2008 Subject: Building an access point with FreeBSD Message-ID: <4919BEE6.7030702@kukulies.org> I have an USB Belkin 802.11g WLAN card and would like to build an access point from it using FreeBSD 7.1. The device is rum0 and it seems to be recognized by the kernel. What is the control program for this interface? ancontrol doesn't seem to work. wicontrol isn't there at all on my system. -- Christoph From ertr1013 at student.uu.se Tue Nov 11 10:58:20 2008 From: ertr1013 at student.uu.se (Erik Trulsson) Date: Tue Nov 11 10:58:27 2008 Subject: Building an access point with FreeBSD In-Reply-To: <4919BEE6.7030702@kukulies.org> References: <4919BEE6.7030702@kukulies.org> Message-ID: <20081111184300.GA22458@owl.midgard.homeip.net> On Tue, Nov 11, 2008 at 06:20:38PM +0100, Christoph Kukulies wrote: > I have an USB Belkin 802.11g WLAN card and would like to build an access > point from it > using FreeBSD 7.1. The device is rum0 and it seems to be recognized by > the kernel. > > What is the control program for this interface? ancontrol doesn't seem > to work. wicontrol isn't there > at all on my system. ifconfig(8) is used for configuring all network interfaces, wireless as well as wired. See also the wlan(4) and rum(4) manpages. -- Erik Trulsson ertr1013@student.uu.se From info at martenvijn.nl Tue Nov 11 13:55:02 2008 From: info at martenvijn.nl (Marten Vijn) Date: Tue Nov 11 13:55:09 2008 Subject: Building an access point with FreeBSD In-Reply-To: <4919BEE6.7030702@kukulies.org> References: <4919BEE6.7030702@kukulies.org> Message-ID: <1226439793.6382.2.camel@mvn-desktop> On Tue, 2008-11-11 at 18:20 +0100, Christoph Kukulies wrote: > I have an USB Belkin 802.11g WLAN card and would like to build an access > point from it > using FreeBSD 7.1. The device is rum0 and it seems to be recognized by > the kernel. > What is the control program for this interface? ifconfig rum0 10.0.0.1/24 mediaopt hostap ssid bert channel 1 if the device supports it. man ifconfig for many more details cheers Marten > ancontrol doesn't seem > to work. wicontrol isn't there > at all on my system. > > -- > Christoph > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Marten Vijn http://martenvijn.nl http://opencommunitycamp.org http://wifisoft.org From jroberson at jroberson.net Tue Nov 11 13:00:51 2008 From: jroberson at jroberson.net (Jeff Roberson) Date: Tue Nov 11 14:12:19 2008 Subject: Typo in ULE in FreeBSD 8.0 -- that's not really a bug In-Reply-To: <6D5D25EA3941074EB7734E51B16687040AD02A77@orsmsx506.amr.corp.intel.com> References: <6D5D25EA3941074EB7734E51B16687040AD02A77@orsmsx506.amr.corp.intel.com> Message-ID: <20081111103600.G964@desktop> On Mon, 10 Nov 2008, Murty, Ravi wrote: > Hello All, > > I have been playing with ULE in 8.0 and while staring at tdq_notify noticed an interesting (and what seems like a typo) problem. > The intention of the function is obvious, send an IPI to notify the remote CPU of some new piece of work. In the case where there is no IPI currently pending on the target CPU and this thread should be preempting what's running there, the code checks in td (passed in as a parameter) is the IDLE thread (TDF_IDLETD). If so, it checks the state and sees if idle is RUNNING and if so figures it will notice this new work and we don't really need to send an expensive IPI. However, why would td (parameter) ever be the IDLE thread? It almost seems like this check will always fail and we end up sending a hard IPI to the target CPU which works, but may not be needed. May be we wanted to use PCPU->curthread instead of td? Wow ravi, thanks. That's what it was at one point. It must've been refactored into brokenness. I'll fix and test soon. This has probably reduced the effectiveness of the mwait patch. Thanks, Jeff > > Thanks > Ravi > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From kuku at kukulies.org Wed Nov 12 03:31:05 2008 From: kuku at kukulies.org (Christoph Kukulies) Date: Wed Nov 12 03:31:14 2008 Subject: Building an access point with FreeBSD In-Reply-To: <1226439793.6382.2.camel@mvn-desktop> References: <4919BEE6.7030702@kukulies.org> <1226439793.6382.2.camel@mvn-desktop> Message-ID: <491ABE73.8000708@kukulies.org> Marten Vijn schrieb: > On Tue, 2008-11-11 at 18:20 +0100, Christoph Kukulies wrote: > >> I have an USB Belkin 802.11g WLAN card and would like to build an access >> point from it >> using FreeBSD 7.1. The device is rum0 and it seems to be recognized by >> the kernel. >> > > >> What is the control program for this interface? >> > > ifconfig rum0 10.0.0.1/24 mediaopt hostap ssid bert channel 1 > > if the device supports it. > Fantastic. So easy! And it works. -- Christoph > man ifconfig for many more details > > cheers > Marten > From avg at icyb.net.ua Wed Nov 12 03:36:50 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Nov 12 03:36:59 2008 Subject: ukbd attachment and root mount In-Reply-To: <4911BA93.9030006@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> Message-ID: <491ABFCD.3060309@icyb.net.ua> on 05/11/2008 17:24 Andriy Gapon said the following: > System is FreeBSD 7.1-BETA2 amd64. > > Looking through my dmesg I see that relative order of ukbd attachment > and root mounting is not deterministic. Sometime keyboard is attached > first, sometimes root filesystem is mounted first. Quite more often root > is mounted first, though. > Example (with GENERIC kernel): > Nov 3 15:40:54 kernel: Trying to mount root from ufs:/dev/mirror/bootgm > Nov 3 15:40:54 kernel: GEOM_LABEL: Label ufs/bootfs removed. > Nov 3 15:40:54 kernel: GEOM_LABEL: Label for provider mirror/bootgm is > ufs/bootfs. > Nov 3 15:40:54 kernel: GEOM_LABEL: Label ufs/bootfs removed. > Nov 3 15:40:54 kernel: ukbd0: 1.10/1.10, addr 3> on uhub2 > Nov 3 15:40:54 kernel: kbd2 at ukbd0 > Nov 3 15:40:54 kernel: uhid0: 1.10/1.10, addr 3> on uhub2 > > Another (with custom kernel, zfs root): > Nov 4 17:54:03 odyssey kernel: Trying to mount root from zfs:tank/root > Nov 4 17:54:03 odyssey kernel: ukbd0: rev 1.10/1.10, addr 3> on uhub2 > Nov 4 17:54:03 odyssey kernel: kbd2 at ukbd0 > Nov 4 17:54:03 odyssey kernel: kbd2: ukbd0, generic (0), config:0x0, > flags:0x3d0000 > Nov 4 17:54:03 odyssey kernel: uhid0: rev 1.10/1.10, addr 3> on uhub2 > > I have a legacy-free system (no PS/2 ports, only USB) and I wanted to > try a kernel without atkbd and psm (with ums, ukbd, kbdmux), but was > bitten hard when I made a mistake and kernel could not find/mount root > filesystem. > > So I stuck at mountroot prompt without a keyboard to enter anything. > This was repeatable about 10 times after which I resorted to live cd. > > Since then I put back atkbdc into my kernel. I guess BIOS or USB > hardware emulate AT or PS/2 keyboard, so the USB keyboard works before > the driver attaches. I guess I need such emulation e.g. for loader or > boot0 configuration. But I guess I don't have to have atkbd driver in > kernel. This turned out not to be a complete solution as it seems that there are some quirks about legacy USB here, sometimes keyboard stops working even at loader prompt (this is described in a different thread). ukbd attachment still puzzles me a lot. I look at some older dmesg, e.g. this 7.0-RELEASE one: http://www.mavetju.org/mail/view_message.php?list=freebsd-usb&id=2709973 and see that ukbd attaches along with ums before mountroot. I look at newer dmesg and I see that ums attaches at about the same time as before but ukbd consistently attaches after mountroot. I wonder what might cause such behavior and how to fix it. I definitely would like to see ukbd attach before mountroot, I can debug this issue, but need some hints on where to start. -- Andriy Gapon From neldredge at math.ucsd.edu Wed Nov 12 03:53:52 2008 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Wed Nov 12 03:53:59 2008 Subject: ukbd attachment and root mount In-Reply-To: <491ABFCD.3060309@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> Message-ID: On Wed, 12 Nov 2008, Andriy Gapon wrote: > on 05/11/2008 17:24 Andriy Gapon said the following: [...] >> I have a legacy-free system (no PS/2 ports, only USB) and I wanted to >> try a kernel without atkbd and psm (with ums, ukbd, kbdmux), but was >> bitten hard when I made a mistake and kernel could not find/mount root >> filesystem. >> >> So I stuck at mountroot prompt without a keyboard to enter anything. >> This was repeatable about 10 times after which I resorted to live cd. >> >> Since then I put back atkbdc into my kernel. I guess BIOS or USB >> hardware emulate AT or PS/2 keyboard, so the USB keyboard works before >> the driver attaches. I guess I need such emulation e.g. for loader or >> boot0 configuration. But I guess I don't have to have atkbd driver in >> kernel. > > This turned out not to be a complete solution as it seems that there are > some quirks about legacy USB here, sometimes keyboard stops working even > at loader prompt (this is described in a different thread). > > ukbd attachment still puzzles me a lot. > I look at some older dmesg, e.g. this 7.0-RELEASE one: > http://www.mavetju.org/mail/view_message.php?list=freebsd-usb&id=2709973 > and see that ukbd attaches along with ums before mountroot. > > I look at newer dmesg and I see that ums attaches at about the same time > as before but ukbd consistently attaches after mountroot. > I wonder what might cause such behavior and how to fix it. > I definitely would like to see ukbd attach before mountroot, I can debug > this issue, but need some hints on where to start. I haven't been following this thread, and I'm pretty sleepy right now, so sorry if this is irrelevant, but I had a somewhat similar problem that was fixed by adding hint.atkbd.0.flags="0x1" to /boot/device.hints . -- Nate Eldredge neldredge@math.ucsd.edu From avg at icyb.net.ua Wed Nov 12 04:00:21 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Nov 12 04:00:34 2008 Subject: ukbd attachment and root mount In-Reply-To: References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> Message-ID: <491AC502.9000507@icyb.net.ua> on 12/11/2008 13:53 Nate Eldredge said the following: > On Wed, 12 Nov 2008, Andriy Gapon wrote: > >> on 05/11/2008 17:24 Andriy Gapon said the following: > [...] >>> I have a legacy-free system (no PS/2 ports, only USB) and I wanted to >>> try a kernel without atkbd and psm (with ums, ukbd, kbdmux), but was >>> bitten hard when I made a mistake and kernel could not find/mount root >>> filesystem. >>> >>> So I stuck at mountroot prompt without a keyboard to enter anything. >>> This was repeatable about 10 times after which I resorted to live cd. >>> >>> Since then I put back atkbdc into my kernel. I guess BIOS or USB >>> hardware emulate AT or PS/2 keyboard, so the USB keyboard works before >>> the driver attaches. I guess I need such emulation e.g. for loader or >>> boot0 configuration. But I guess I don't have to have atkbd driver in >>> kernel. >> >> This turned out not to be a complete solution as it seems that there are >> some quirks about legacy USB here, sometimes keyboard stops working even >> at loader prompt (this is described in a different thread). >> >> ukbd attachment still puzzles me a lot. >> I look at some older dmesg, e.g. this 7.0-RELEASE one: >> http://www.mavetju.org/mail/view_message.php?list=freebsd-usb&id=2709973 >> and see that ukbd attaches along with ums before mountroot. >> >> I look at newer dmesg and I see that ums attaches at about the same time >> as before but ukbd consistently attaches after mountroot. >> I wonder what might cause such behavior and how to fix it. >> I definitely would like to see ukbd attach before mountroot, I can debug >> this issue, but need some hints on where to start. > > I haven't been following this thread, and I'm pretty sleepy right now, > so sorry if this is irrelevant, but I had a somewhat similar problem > that was fixed by adding > > hint.atkbd.0.flags="0x1" > > to /boot/device.hints . > I can try this, but I think this wouldn't help for two reasons: 1. I already tried kernel without atkb at all 2. if ukbd driver is not attached then I don't see any way USB keyboard would work in non-legacy way Anyway I will try this, thank you. -- Andriy Gapon From koitsu at FreeBSD.org Wed Nov 12 04:14:12 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Wed Nov 12 04:14:25 2008 Subject: ukbd attachment and root mount In-Reply-To: <491AC502.9000507@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> Message-ID: <20081112121410.GA24629@icarus.home.lan> On Wed, Nov 12, 2008 at 01:58:58PM +0200, Andriy Gapon wrote: > on 12/11/2008 13:53 Nate Eldredge said the following: > > On Wed, 12 Nov 2008, Andriy Gapon wrote: > > > >> on 05/11/2008 17:24 Andriy Gapon said the following: > > [...] > >>> I have a legacy-free system (no PS/2 ports, only USB) and I wanted to > >>> try a kernel without atkbd and psm (with ums, ukbd, kbdmux), but was > >>> bitten hard when I made a mistake and kernel could not find/mount root > >>> filesystem. > >>> > >>> So I stuck at mountroot prompt without a keyboard to enter anything. > >>> This was repeatable about 10 times after which I resorted to live cd. > >>> > >>> Since then I put back atkbdc into my kernel. I guess BIOS or USB > >>> hardware emulate AT or PS/2 keyboard, so the USB keyboard works before > >>> the driver attaches. I guess I need such emulation e.g. for loader or > >>> boot0 configuration. But I guess I don't have to have atkbd driver in > >>> kernel. > >> > >> This turned out not to be a complete solution as it seems that there are > >> some quirks about legacy USB here, sometimes keyboard stops working even > >> at loader prompt (this is described in a different thread). > >> > >> ukbd attachment still puzzles me a lot. > >> I look at some older dmesg, e.g. this 7.0-RELEASE one: > >> http://www.mavetju.org/mail/view_message.php?list=freebsd-usb&id=2709973 > >> and see that ukbd attaches along with ums before mountroot. > >> > >> I look at newer dmesg and I see that ums attaches at about the same time > >> as before but ukbd consistently attaches after mountroot. > >> I wonder what might cause such behavior and how to fix it. > >> I definitely would like to see ukbd attach before mountroot, I can debug > >> this issue, but need some hints on where to start. > > > > I haven't been following this thread, and I'm pretty sleepy right now, > > so sorry if this is irrelevant, but I had a somewhat similar problem > > that was fixed by adding > > > > hint.atkbd.0.flags="0x1" > > > > to /boot/device.hints . To those reading, the above setting enables the following option: bit 0 (FAIL_IF_NO_KBD) By default the atkbd driver will install even if a keyboard is not actually connected to the system. This option prevents the driver from being installed in this situation. > I can try this, but I think this wouldn't help for two reasons: > 1. I already tried kernel without atkb at all > 2. if ukbd driver is not attached then I don't see any way USB keyboard > would work in non-legacy way Regarding #2: at which stage? boot0/boot2/loader require an AT or PS/2 keyboard to work. None of these stages use ukbd(4) or anything -- there is no kernel loaded at this point!! Meaning: if you have a USB keyboard, your BIOS will need to have a "USB Legacy" option to cause it to act as a PS/2 keyboard, for typing in boot0/boot2/loader to work. Device hints are for kernel drivers, once the kernel is loaded. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From avg at icyb.net.ua Wed Nov 12 04:20:46 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Nov 12 04:20:58 2008 Subject: ukbd attachment and root mount In-Reply-To: <20081112121410.GA24629@icarus.home.lan> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> Message-ID: <491ACA19.2040008@icyb.net.ua> on 12/11/2008 14:14 Jeremy Chadwick said the following: > On Wed, Nov 12, 2008 at 01:58:58PM +0200, Andriy Gapon wrote: [snip] >> 2. if ukbd driver is not attached then I don't see any way USB keyboard >> would work in non-legacy way > > Regarding #2: at which stage? boot0/boot2/loader require an AT or PS/2 > keyboard to work. None of these stages use ukbd(4) or anything -- there > is no kernel loaded at this point!! Meaning: if you have a USB keyboard, > your BIOS will need to have a "USB Legacy" option to cause it to act as > a PS/2 keyboard, for typing in boot0/boot2/loader to work. > > Device hints are for kernel drivers, once the kernel is loaded. Jeremy, I understand all of this. In subject line and earlier messages I say that I am interested in mountroot prompt - the prompt where kernel can ask about what device to use for root filesystem. Essentially I would like kernel to recognize USB keyboard (and disable all the legacy stuff if needed) before it prompts for the root device. -- Andriy Gapon From koitsu at FreeBSD.org Wed Nov 12 04:33:17 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Wed Nov 12 04:33:29 2008 Subject: ukbd attachment and root mount In-Reply-To: <491ACA19.2040008@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> Message-ID: <20081112123315.GA24907@icarus.home.lan> On Wed, Nov 12, 2008 at 02:20:41PM +0200, Andriy Gapon wrote: > on 12/11/2008 14:14 Jeremy Chadwick said the following: > > On Wed, Nov 12, 2008 at 01:58:58PM +0200, Andriy Gapon wrote: > [snip] > >> 2. if ukbd driver is not attached then I don't see any way USB keyboard > >> would work in non-legacy way > > > > Regarding #2: at which stage? boot0/boot2/loader require an AT or PS/2 > > keyboard to work. None of these stages use ukbd(4) or anything -- there > > is no kernel loaded at this point!! Meaning: if you have a USB keyboard, > > your BIOS will need to have a "USB Legacy" option to cause it to act as > > a PS/2 keyboard, for typing in boot0/boot2/loader to work. > > > > Device hints are for kernel drivers, once the kernel is loaded. > > Jeremy, > > I understand all of this. > In subject line and earlier messages I say that I am interested in > mountroot prompt - the prompt where kernel can ask about what device to > use for root filesystem. > Essentially I would like kernel to recognize USB keyboard (and disable > all the legacy stuff if needed) before it prompts for the root device. I fully understand that fact. However, I don't see the logic in that statement. You should be able to remove and add a keyboard at any time and be able to type immediately. Meaning: I don't see why when the keyboard recognition is performed (e.g. before printing mountroot or after) matters. It should not. I think this is a red herring. I've seen the problem where I have a fully functional USB keyboard in boot0/boot2/loader and in multi-user, but when booting into single-user or when getting a mountroot prompt, the keyboard does not function. When the mountroot prompt is printed (before or after ukbd attached) makes no difference for me in this scenario -- I tested it many times. It's very possible that "something" (kbdcontrol?) is getting run only during late stages of multi-user, which makes the keyboard work. But prior to that "something" being run (but AFTER boot2/loader), the keyboard is not truly usable. I hope everyone here is also aware of that fact that not all keyboards are created equal. Case in point (and this reason is exactly why I am purchasing a native PS/2 keyboard, as USB4BSD doesn't work with all USB keyboards right now): http://lists.freebsd.org/pipermail/freebsd-current/2008-November/000219.html The bottom line: FreeBSD cannot be reliably used with a USB keyboard in all circumstances. And that is a very sad reality, because 90% of the keyboards you find on the consumer and enterprise market are USB -- native PS/2 keyboards are now a scarcity. Do not even for a minute tell me "buy a USB-to-PS2 adapter", because the "green ones" that come with USB mice do not work with USB keyboards. I have even bought a "purple" USB-to-PS2 keyboard adapter from Amazon, specifically for this purpose, and it *does not work*. I found out weeks later the adapters only work on CERTAIN models of USB keyboards, depending upon how they're engineered. What really needs to happen here should be obvious: we need some form of inexpensive keyboard-only USB support in boot2/loader. I would *love* to know how Linux and Windows solve this problem. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From alex at metrocom.ru Wed Nov 12 04:40:27 2008 From: alex at metrocom.ru (Varshavchick Alexander) Date: Wed Nov 12 04:40:33 2008 Subject: FreeBSD 5.4 - filesystem full Message-ID: <20081112145105.T65116@comanche.metrocom.ru> I have an old enough server with FreeBSD 5.4 which from time to time complains about filesystem full. But the problem is that the partition in question has about 15G free space and more than 10000000 free inodes. Then all by itself the error dissapears, only to be repeated several hours later. What can it be and where to look? The server runs mainly apache and sendmail, nothing special. Thanks and regards ---- Alexander Varshavchick, Metrocom Joint Stock Company Phone: (812)718-3322, 718-3115(fax) From avg at icyb.net.ua Wed Nov 12 04:49:20 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Nov 12 04:50:31 2008 Subject: ukbd attachment and root mount In-Reply-To: <20081112123315.GA24907@icarus.home.lan> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> Message-ID: <491AD0CB.8050309@icyb.net.ua> on 12/11/2008 14:33 Jeremy Chadwick said the following: > On Wed, Nov 12, 2008 at 02:20:41PM +0200, Andriy Gapon wrote: >> on 12/11/2008 14:14 Jeremy Chadwick said the following: >>> On Wed, Nov 12, 2008 at 01:58:58PM +0200, Andriy Gapon wrote: >> [snip] >>>> 2. if ukbd driver is not attached then I don't see any way USB keyboard >>>> would work in non-legacy way >>> Regarding #2: at which stage? boot0/boot2/loader require an AT or PS/2 >>> keyboard to work. None of these stages use ukbd(4) or anything -- there >>> is no kernel loaded at this point!! Meaning: if you have a USB keyboard, >>> your BIOS will need to have a "USB Legacy" option to cause it to act as >>> a PS/2 keyboard, for typing in boot0/boot2/loader to work. >>> >>> Device hints are for kernel drivers, once the kernel is loaded. >> Jeremy, >> >> I understand all of this. >> In subject line and earlier messages I say that I am interested in >> mountroot prompt - the prompt where kernel can ask about what device to >> use for root filesystem. >> Essentially I would like kernel to recognize USB keyboard (and disable >> all the legacy stuff if needed) before it prompts for the root device. > > I fully understand that fact. However, I don't see the logic in that > statement. You should be able to remove and add a keyboard at any time > and be able to type immediately. Meaning: I don't see why when the > keyboard recognition is performed (e.g. before printing mountroot or > after) matters. It should not. I think this is a red herring. I think that this does matter because keyboard recognition is performed after the 'mounting from' log line *only if* root mount is done automatically. If there is an actual interactive prompt then recognition is not performed, at least I do not see any relevant lines on the screen and I am stuck at the prompt. > I've seen the problem where I have a fully functional USB keyboard in > boot0/boot2/loader For me it even randomly dies at these stages. I reported this in a different thread. But this should not be related to kernel behavior. >and in multi-user, For me this always works. > but when booting into single-user For me this always works. > or when getting a mountroot prompt, the keyboard does not function. > When the mountroot prompt is printed (before or after ukbd attached) > makes no difference for me in this scenario -- I tested it many times. For me ukbd lines are never printed if I get actual interactive mountroot prompt. > It's very possible that "something" (kbdcontrol?) is getting run only > during late stages of multi-user, which makes the keyboard work. But > prior to that "something" being run (but AFTER boot2/loader), the > keyboard is not truly usable. For me this is not true. My keyboard always works after ukbd lines appear on screen. > I hope everyone here is also aware of that fact that not all keyboards > are created equal. Case in point (and this reason is exactly why I > am purchasing a native PS/2 keyboard, as USB4BSD doesn't work with > all USB keyboards right now): For me this is not an option, no PS/2 ports. > http://lists.freebsd.org/pipermail/freebsd-current/2008-November/000219.html > > The bottom line: > > FreeBSD cannot be reliably used with a USB keyboard in all > circumstances.And that is a very sad reality, because 90% of the > keyboards you find on the consumer and enterprise market are USB -- > native PS/2 keyboards are now a scarcity. I agree that this is a sad reality but only for boot stages where we depend on external entity named BIOS to help us. This doesn't have to be a sad reality once kernel takes control. USB support in boot chain - I don't know - this would be great of course but that's a lot of code. -- Andriy Gapon From jhb at freebsd.org Wed Nov 12 05:11:50 2008 From: jhb at freebsd.org (John Baldwin) Date: Wed Nov 12 05:12:10 2008 Subject: uname -c: alias for uname -rms In-Reply-To: <49179E9A.2020507@gmail.com> References: <491764C3.5020504@gmail.com> <491782BD.9090009@freebsd.org> <49179E9A.2020507@gmail.com> Message-ID: <200811111132.14779.jhb@freebsd.org> On Sunday 09 November 2008 09:38:18 pm Eitan Adler wrote: > Tim Kientzle wrote: > > Are there any precedents for this option? > Not really - its more like the recent cp -a switch: an alias for a > common set of switches. However, cp -a was added to be compatible with other OS's, not purely as a shortcut. -- John Baldwin From jhb at freebsd.org Wed Nov 12 05:11:55 2008 From: jhb at freebsd.org (John Baldwin) Date: Wed Nov 12 05:12:11 2008 Subject: Typo in ULE in FreeBSD 8.0 -- that's not really a bug In-Reply-To: <6D5D25EA3941074EB7734E51B16687040AD02A77@orsmsx506.amr.corp.intel.com> References: <6D5D25EA3941074EB7734E51B16687040AD02A77@orsmsx506.amr.corp.intel.com> Message-ID: <200811111137.55731.jhb@freebsd.org> On Monday 10 November 2008 12:57:44 pm Murty, Ravi wrote: > Hello All, > > I have been playing with ULE in 8.0 and while staring at tdq_notify noticed an interesting (and what seems like a typo) problem. > The intention of the function is obvious, send an IPI to notify the remote CPU of some new piece of work. In the case where there is no IPI currently pending on the target CPU and this thread should be preempting what's running there, the code checks in td (passed in as a parameter) is the IDLE thread (TDF_IDLETD). If so, it checks the state and sees if idle is RUNNING and if so figures it will notice this new work and we don't really need to send an expensive IPI. However, why would td (parameter) ever be the IDLE thread? It almost seems like this check will always fail and we end up sending a hard IPI to the target CPU which works, but may not be needed. May be we wanted to use PCPU->curthread instead of td? I think you are correct. Something like this might fix it: Index: sched_ule.c =================================================================== RCS file: /usr/cvs/src/sys/kern/sched_ule.c,v retrieving revision 1.246 diff -u -r1.246 sched_ule.c --- sched_ule.c 19 Jul 2008 05:13:47 -0000 1.246 +++ sched_ule.c 11 Nov 2008 16:36:25 -0000 @@ -942,7 +942,7 @@ static void tdq_notify(struct tdq *tdq, struct thread *td) { - int cpri; + struct thread *ctd; int pri; int cpu; @@ -950,10 +950,10 @@ return; cpu = td->td_sched->ts_cpu; pri = td->td_priority; - cpri = pcpu_find(cpu)->pc_curthread->td_priority; - if (!sched_shouldpreempt(pri, cpri, 1)) + ctd = pcpu_find(cpu)->pc_curthread; + if (!sched_shouldpreempt(pri, ctd->td_priority, 1)) return; - if (TD_IS_IDLETHREAD(td)) { + if (TD_IS_IDLETHREAD(ctd)) { /* * If the idle thread is still 'running' it's probably * waiting on us to release the tdq spinlock already. No -- John Baldwin From koitsu at FreeBSD.org Wed Nov 12 05:21:26 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Wed Nov 12 05:21:39 2008 Subject: ukbd attachment and root mount In-Reply-To: <491AD0CB.8050309@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> <491AD0CB.8050309@icyb.net.ua> Message-ID: <20081112132124.GA25637@icarus.home.lan> On Wed, Nov 12, 2008 at 02:49:15PM +0200, Andriy Gapon wrote: > on 12/11/2008 14:33 Jeremy Chadwick said the following: > > On Wed, Nov 12, 2008 at 02:20:41PM +0200, Andriy Gapon wrote: > >> on 12/11/2008 14:14 Jeremy Chadwick said the following: > >>> On Wed, Nov 12, 2008 at 01:58:58PM +0200, Andriy Gapon wrote: > >> [snip] > >>>> 2. if ukbd driver is not attached then I don't see any way USB keyboard > >>>> would work in non-legacy way > >>> Regarding #2: at which stage? boot0/boot2/loader require an AT or PS/2 > >>> keyboard to work. None of these stages use ukbd(4) or anything -- there > >>> is no kernel loaded at this point!! Meaning: if you have a USB keyboard, > >>> your BIOS will need to have a "USB Legacy" option to cause it to act as > >>> a PS/2 keyboard, for typing in boot0/boot2/loader to work. > >>> > >>> Device hints are for kernel drivers, once the kernel is loaded. > >> Jeremy, > >> > >> I understand all of this. > >> In subject line and earlier messages I say that I am interested in > >> mountroot prompt - the prompt where kernel can ask about what device to > >> use for root filesystem. > >> Essentially I would like kernel to recognize USB keyboard (and disable > >> all the legacy stuff if needed) before it prompts for the root device. > > > > I fully understand that fact. However, I don't see the logic in that > > statement. You should be able to remove and add a keyboard at any time > > and be able to type immediately. Meaning: I don't see why when the > > keyboard recognition is performed (e.g. before printing mountroot or > > after) matters. It should not. I think this is a red herring. > > I think that this does matter because keyboard recognition is performed > after the 'mounting from' log line *only if* root mount is done > automatically. > If there is an actual interactive prompt then recognition is not > performed, at least I do not see any relevant lines on the screen and I > am stuck at the prompt. > > > I've seen the problem where I have a fully functional USB keyboard in > > boot0/boot2/loader > > For me it even randomly dies at these stages. > I reported this in a different thread. > But this should not be related to kernel behavior. > > >and in multi-user, > > For me this always works. > > > but when booting into single-user > > For me this always works. > > > or when getting a mountroot prompt, the keyboard does not function. > > When the mountroot prompt is printed (before or after ukbd attached) > > makes no difference for me in this scenario -- I tested it many times. > > For me ukbd lines are never printed if I get actual interactive > mountroot prompt. > > > It's very possible that "something" (kbdcontrol?) is getting run only > > during late stages of multi-user, which makes the keyboard work. But > > prior to that "something" being run (but AFTER boot2/loader), the > > keyboard is not truly usable. > > For me this is not true. My keyboard always works after ukbd lines > appear on screen. I've pointed you to evidence where this isn't true, especially when using the USB4BSD stack. There is something called "boot legacy protocol" which USB keyboards have to support to properly be interfaced with in FreeBSD using the USB4BSD stack; in the case of the Microsoft Natural Ergo 4000 keyboard, it does not play well with USB4BSD (it DOES work with the old USB stack, but none of the multimedia keys work, and worse, the F-Lock key does not work; this is because those keys use uhid(4) and not ukbd(4)). Linux has a __20 page Wiki document__ on **just this keyboard**. That should give you some idea of how complex the situation with USB keyboards is in general. http://www.gentoo-wiki.info/HOWTO_Microsoft_Natural_Ergonomic_Keyboard_4000 > > I hope everyone here is also aware of that fact that not all keyboards > > are created equal. Case in point (and this reason is exactly why I > > am purchasing a native PS/2 keyboard, as USB4BSD doesn't work with > > all USB keyboards right now): > > For me this is not an option, no PS/2 ports. I don't know what to say to ***ANY*** of the above, other than this: No one is doing anything about this problem because there does not appear to be a 100% reproducible always-screws-up-when-I-do-this scenario that happens to *every FreeBSD user*. Until we settle down, stop replying to Emails with one-liner injections, and compile a list of test scenarios/cases that people can perform, and get these people to provide both 1) full hardware details, 2) full kernel configuration files, 3) full loader.conf files, and 4) full device.hints files, we're not going to get anywhere. > > http://lists.freebsd.org/pipermail/freebsd-current/2008-November/000219.html > > > > The bottom line: > > > > FreeBSD cannot be reliably used with a USB keyboard in all > > circumstances.And that is a very sad reality, because 90% of the > > keyboards you find on the consumer and enterprise market are USB -- > > native PS/2 keyboards are now a scarcity. > > I agree that this is a sad reality but only for boot stages where we > depend on external entity named BIOS to help us. > This doesn't have to be a sad reality once kernel takes control. It's been confirmed by numerous people now, including #bsdports users, that "USB Legacy" does not work for some individuals. This is either because of BIOS bugs, or because the USB keyboards do not support tying into SMM. We don't know the true cause. One thing we do know: we have FreeBSD users stating they cannot type in boot0/boot2/loader, even with USB Legacy enabled, so going into single-user after a reboot is impossible. Another thing we do know: we have FreeBSD users who do not have fully functional USB keyboards in FreeBSD (some see ukbd attach, others do not; some are using USB4BSD, others are not). So, can someone take the time to come up with test scenarios/cases so that users can perform these tests, list off the exact hardware they have, and we can see if there is a consistent/common failure between everyone? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From avg at icyb.net.ua Wed Nov 12 05:33:52 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Wed Nov 12 05:34:11 2008 Subject: ukbd attachment and root mount In-Reply-To: <20081112132124.GA25637@icarus.home.lan> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> <491AD0CB.8050309@icyb.net.ua> <20081112132124.GA25637@icarus.home.lan> Message-ID: <491ADB3B.2090000@icyb.net.ua> on 12/11/2008 15:21 Jeremy Chadwick said the following: > I don't know what to say to ***ANY*** of the above, other than this: > > No one is doing anything about this problem because there does not > appear to be a 100% reproducible always-screws-up-when-I-do-this > scenario that happens to *every FreeBSD user*. > > Until we settle down, stop replying to Emails with one-liner injections, > and compile a list of test scenarios/cases that people can perform, and > get these people to provide both 1) full hardware details, 2) full > kernel configuration files, 3) full loader.conf files, and 4) full > device.hints files, we're not going to get anywhere. Well I started two separate threads. This thread is about one very specific issue - ukbd attaching after mountroot code. Again, in this thread I am only interested in getting ukbd to attach before the mount root. I am not interested in BIOS, boot chain, etc. I am not even interested in speculations about whether keyboard would work or not at mountroot prompt if it were attaching before it. -- Andriy Gapon From ivoras at freebsd.org Wed Nov 12 07:05:00 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Wed Nov 12 07:05:13 2008 Subject: FreeBSD 5.4 - filesystem full In-Reply-To: <20081112145105.T65116@comanche.metrocom.ru> References: <20081112145105.T65116@comanche.metrocom.ru> Message-ID: Varshavchick Alexander wrote: > I have an old enough server with FreeBSD 5.4 which from time to time > complains about filesystem full. But the problem is that the partition > in question has about 15G free space and more than 10000000 free inodes. > Then all by itself the error dissapears, only to be repeated several > hours later. What can it be and where to look? The server runs mainly > apache and sendmail, nothing special. I don't know for sure, but here's some generic troubleshooting: a) Are you 100% sure there isn't an application that periodically fills the drive? This would be easiest to solve - all other problems are worse. b) How is your IO rate at the time you run out of space? c) Did you try fsck-ing the file system? d) Why can't you upgrade to a more recent version of FreeBSD, like 7.1? From ravi.murty at intel.com Wed Nov 12 08:09:44 2008 From: ravi.murty at intel.com (Murty, Ravi) Date: Wed Nov 12 08:09:51 2008 Subject: Typo in ULE in FreeBSD 8.0 -- that's not really a bug In-Reply-To: <200811111137.55731.jhb@freebsd.org> References: <6D5D25EA3941074EB7734E51B16687040AD02A77@orsmsx506.amr.corp.intel.com> <200811111137.55731.jhb@freebsd.org> Message-ID: <6D5D25EA3941074EB7734E51B16687040AD518AC@orsmsx506.amr.corp.intel.com> Yes, that's what I was thinking. Just look at what's running on the remote CPU. Thanks Jeff and John. Ravi -----Original Message----- From: John Baldwin [mailto:jhb@freebsd.org] Sent: Tuesday, November 11, 2008 8:38 AM To: freebsd-hackers@freebsd.org Cc: Murty, Ravi; jeff@freebsd.org Subject: Re: Typo in ULE in FreeBSD 8.0 -- that's not really a bug On Monday 10 November 2008 12:57:44 pm Murty, Ravi wrote: > Hello All, > > I have been playing with ULE in 8.0 and while staring at tdq_notify noticed an interesting (and what seems like a typo) problem. > The intention of the function is obvious, send an IPI to notify the remote CPU of some new piece of work. In the case where there is no IPI currently pending on the target CPU and this thread should be preempting what's running there, the code checks in td (passed in as a parameter) is the IDLE thread (TDF_IDLETD). If so, it checks the state and sees if idle is RUNNING and if so figures it will notice this new work and we don't really need to send an expensive IPI. However, why would td (parameter) ever be the IDLE thread? It almost seems like this check will always fail and we end up sending a hard IPI to the target CPU which works, but may not be needed. May be we wanted to use PCPU->curthread instead of td? I think you are correct. Something like this might fix it: Index: sched_ule.c =================================================================== RCS file: /usr/cvs/src/sys/kern/sched_ule.c,v retrieving revision 1.246 diff -u -r1.246 sched_ule.c --- sched_ule.c 19 Jul 2008 05:13:47 -0000 1.246 +++ sched_ule.c 11 Nov 2008 16:36:25 -0000 @@ -942,7 +942,7 @@ static void tdq_notify(struct tdq *tdq, struct thread *td) { - int cpri; + struct thread *ctd; int pri; int cpu; @@ -950,10 +950,10 @@ return; cpu = td->td_sched->ts_cpu; pri = td->td_priority; - cpri = pcpu_find(cpu)->pc_curthread->td_priority; - if (!sched_shouldpreempt(pri, cpri, 1)) + ctd = pcpu_find(cpu)->pc_curthread; + if (!sched_shouldpreempt(pri, ctd->td_priority, 1)) return; - if (TD_IS_IDLETHREAD(td)) { + if (TD_IS_IDLETHREAD(ctd)) { /* * If the idle thread is still 'running' it's probably * waiting on us to release the tdq spinlock already. No -- John Baldwin From babkin at verizon.net Wed Nov 12 06:08:12 2008 From: babkin at verizon.net (Sergey Babkin) Date: Wed Nov 12 08:47:36 2008 Subject: FreeBSD 5.4 - filesystem full References: <20081112145105.T65116@comanche.metrocom.ru> Message-ID: <491AD6DF.24F8072F@verizon.net> Varshavchick Alexander wrote: > > I have an old enough server with FreeBSD 5.4 which from time to time > complains about filesystem full. But the problem is that the partition > in question has about 15G free space and more than 10000000 free inodes. > Then all by itself the error dissapears, only to be repeated several hours > later. What can it be and where to look? The server runs mainly apache and > sendmail, nothing special. I vaguely remember that there was an issue with softupdates that didn't report blocks as free until the filesystem was synced, and with intense disk activity the filesystem was not syncing by itself often enough. -SB From babkin at verizon.net Wed Nov 12 06:11:53 2008 From: babkin at verizon.net (Sergey Babkin) Date: Wed Nov 12 08:47:44 2008 Subject: ukbd attachment and root mount References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> Message-ID: <491AD7BB.2EAA9AA0@verizon.net> Jeremy Chadwick wrote: > > What really needs to happen here should be obvious: we need some form of > inexpensive keyboard-only USB support in boot2/loader. > > I would *love* to know how Linux and Windows solve this problem. If I remember right, UnixWare used(s) the BIOS calls in the loader. -SB From marc.loerner at hob.de Wed Nov 12 09:43:43 2008 From: marc.loerner at hob.de (Marc =?iso-8859-1?q?L=F6rner?=) Date: Wed Nov 12 09:48:52 2008 Subject: ide with DMA and ram > 4GB Message-ID: <200811121823.14400.marc.loerner@hob.de> Hello, I just stepped over a problem with my IDE disk running in DMA-mode and having more than 4GB of RAM. It seems that the whole way down GEOM, ata-disk, ata-dma never is checked whether physical address of buffer is less than 4GB an so fits in 32bits. => when PRD is set the address is rigorously truncated to fit into 32bit, with buffer < 4GB all is quite fine. So what do you recommend: - Easiest way (but not performantest) is to turn DMA for ide off and use PIO instead - Harder way: Bugfix ide-dma! When doing this, where do you recommend to put code? I tried to do a simple (but not performant) patch in in ata-disk functions ad_strategy and ad_done with using another aligned, with right boundary and physical address < 4GB and copying data from an to it from bp->b_data (or request->data). At least this one works, but copying from one to another buffer is not quite elegant and the code doesn't really belong there. So I thought of putting something into busdma_machdep, but right now I have now clue to where to hook in, because this function cannot modify buffer given from vfs_bio => geom => ata. Regards, Marc Loerner P.S.: I'm using kernel-sources from 7.0-RELEASE-p3 From des at des.no Wed Nov 12 09:48:30 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed Nov 12 09:49:43 2008 Subject: ukbd attachment and root mount In-Reply-To: <491AD7BB.2EAA9AA0@verizon.net> (Sergey Babkin's message of "Wed, 12 Nov 2008 08:18:51 -0500") References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> <491AD7BB.2EAA9AA0@verizon.net> Message-ID: <86iqqsx2c2.fsf@ds4.des.no> Sergey Babkin writes: > Jeremy Chadwick writes: > > What really needs to happen here should be obvious: we need some > > form of inexpensive keyboard-only USB support in boot2/loader. > If I remember right, UnixWare used(s) the BIOS calls in the loader. So does FreeBSD. DES -- Dag-Erling Sm?rgrav - des@des.no From ady at freebsd.ady.ro Wed Nov 12 10:16:28 2008 From: ady at freebsd.ady.ro (Adrian Penisoara) Date: Wed Nov 12 10:16:36 2008 Subject: FreeBSD 5.4 - filesystem full In-Reply-To: <20081112145105.T65116@comanche.metrocom.ru> References: <20081112145105.T65116@comanche.metrocom.ru> Message-ID: <78cb3d3f0811120944n5e57cbf5xc1776930aae85c06@mail.gmail.com> Hi, What kind of applications are you running on the machine ? Are they mmap'ing files on the filesystem in quesiton (which one ?) ? AFAIR even if you delete a big file the disk space may not be reclaimed if a process still has the file open. If you reboot the machine or restart some of the applications, does the issue disappear ? Regards, Adrian. On Wed, Nov 12, 2008 at 1:05 PM, Varshavchick Alexander wrote: > I have an old enough server with FreeBSD 5.4 which from time to time > complains about filesystem full. But the problem is that the partition in > question has about 15G free space and more than 10000000 free inodes. Then > all by itself the error dissapears, only to be repeated several hours later. > What can it be and where to look? The server runs mainly apache and > sendmail, nothing special. > > Thanks and regards > > ---- > Alexander Varshavchick, Metrocom Joint Stock Company > Phone: (812)718-3322, 718-3115(fax) > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From nakal at web.de Wed Nov 12 09:49:51 2008 From: nakal at web.de (Martin) Date: Wed Nov 12 10:17:43 2008 Subject: ukbd attachment and root mount In-Reply-To: <20081112132124.GA25637@icarus.home.lan> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> <491AD0CB.8050309@icyb.net.ua> <20081112132124.GA25637@icarus.home.lan> Message-ID: <20081112183012.57af6eb5@zelda.local> Am Wed, 12 Nov 2008 05:21:24 -0800 schrieb Jeremy Chadwick : > Until we settle down, stop replying to Emails with one-liner > injections, and compile a list of test scenarios/cases that people > can perform, and get these people to provide both 1) full hardware > details, 2) full kernel configuration files, 3) full loader.conf > files, and 4) full device.hints files, we're not going to get > anywhere. Ok, I will add the details for the GA-EP45-DS3R based system. 1) dmesg Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.1-PRERELEASE #0: Mon Nov 10 08:23:21 CET 2008 root@kirby:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Duo CPU E8500 @ 3.16GHz (3166.32-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x10676 Stepping = 6 Features=0xbfebfbff Features2=0x8e3fd> AMD Features=0x20100800 AMD Features2=0x1 Cores per package: 2 usable memory = 8574255104 (8177 MB) avail memory = 8286810112 (7902 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0: Changing APIC ID to 2 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, cfdb0000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0xa000-0xa0ff mem 0xd0000000-0xdfffffff,0xe5000000-0xe500ffff irq 16 at device 0.0 on pci1 pcm0: mem 0xe5010000-0xe5013fff irq 17 at device 0.1 on pci1 pcm0: [ITHREAD] uhci0: port 0xe000-0xe01f irq 16 at device 26.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xe100-0xe11f irq 21 at device 26.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xe200-0xe21f irq 18 at device 26.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xe9305000-0xe93053ff irq 18 at device 26.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: on usb3 uhub3: 6 ports with 6 removable, self powered pcm1: mem 0xe9300000-0xe9303fff irq 22 at device 27.0 on pci0 pcm1: [ITHREAD] pcib2: irq 16 at device 28.0 on pci0 pci2: on pcib2 pcib3: irq 19 at device 28.3 on pci0 pci3: on pcib3 atapci0: port 0xb000-0xb007,0xb100-0xb103,0xb200-0xb207,0xb300-0xb303,0xb400-0xb40f irq 19 at device 0.0 on pci3 atapci0: [ITHREAD] ata2: on atapci0 ata2: [ITHREAD] pcib4: irq 16 at device 28.4 on pci0 pci4: on pcib4 re0: port 0xc000-0xc0ff mem Ethernet> 0xe9010000-0xe9010fff,0xe9000000-0xe900ffff irq 16 at device Ethernet> 0.0 on pci4 re0: Chip rev. 0x3c000000 re0: MAC rev. 0x00400000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto re0: Ethernet address: 00:1f:d0:24:96:ab re0: [FILTER] pcib5: irq 17 at device 28.5 on pci0 pci5: on pcib5 re1: port 0xd000-0xd0ff mem Ethernet> 0xe9110000-0xe9110fff,0xe9100000-0xe910ffff irq 17 at device Ethernet> 0.0 on pci5 re1: Chip rev. 0x3c000000 re1: MAC rev. 0x00400000 miibus1: on re1 rgephy1: PHY 1 on miibus1 rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto re1: Ethernet address: 00:1f:d0:24:96:a9 re1: [FILTER] uhci3: port 0xe300-0xe31f irq 23 at device 29.0 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb4: on uhci3 usb4: USB revision 1.0 uhub4: on usb4 uhub4: 2 ports with 2 removable, self powered uhci4: port 0xe400-0xe41f irq 19 at device 29.1 on pci0 uhci4: [GIANT-LOCKED] uhci4: [ITHREAD] usb5: on uhci4 usb5: USB revision 1.0 uhub5: on usb5 uhub5: 2 ports with 2 removable, self powered uhci5: port 0xe500-0xe51f irq 18 at device 29.2 on pci0 uhci5: [GIANT-LOCKED] uhci5: [ITHREAD] usb6: on uhci5 usb6: USB revision 1.0 uhub6: on usb6 uhub6: 2 ports with 2 removable, self powered ehci1: mem 0xe9304000-0xe93043ff irq 23 at device 29.7 on pci0 ehci1: [GIANT-LOCKED] ehci1: [ITHREAD] usb7: EHCI version 1.0 usb7: companion controllers, 2 ports each: usb4 usb5 usb6 usb7: on ehci1 usb7: USB revision 2.0 uhub7: on usb7 uhub7: 6 ports with 6 removable, self powered pcib6: at device 30.0 on pci0 pci6: on pcib6 fwohci0: mem 0xe9204000-0xe92047ff,0xe9200000-0xe9203fff irq 23 at device 7.0 on pci6 fwohci0: [FILTER] fwohci0: OHCI version 1.10 (ROM=0) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:2c:a1:59:00:00:1f:d0 fwohci0: Phy 1394a available S400, 3 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:2c:a1:00:1f:d0 fwe0: Ethernet address: 02:2c:a1:00:1f:d0 fwip0: on firewire0 fwip0: Firewire address: 00:2c:a1:59:00:00:1f:d0 @ 0xfffe00000000, S400, maxrec 2048 sbp0: on firewire0 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0xcf2b4000 fwohci0: Initiate bus reset fwohci0: BUS reset fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode isab0: at device 31.0 on pci0 isa0: on isab0 atapci1: port 0xe600-0xe607,0xe700-0xe703,0xe800-0xe807,0xe900-0xe903,0xea00-0xea1f mem 0xe9306000-0xe93067ff irq 19 at device 31.2 on pci0 atapci1: [ITHREAD] atapci1: AHCI Version 01.20 controller with 6 ports detected ata3: on atapci1 ata3: [ITHREAD] ata4: on atapci1 ata4: [ITHREAD] ata5: on atapci1 ata5: [ITHREAD] ata6: on atapci1 ata6: [ITHREAD] ata7: on atapci1 ata7: [ITHREAD] ata8: on atapci1 ata8: [ITHREAD] pci0: at device 31.3 (no driver attached) fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 ppbus0: [ITHREAD] plip0: on ppbus0 plip0: WARNING: using obsoleted IFF_NEEDSGIANT flag lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] cpu0: on acpi0 est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 61a492006004920 device_attach: est0 attach returned 6 p4tcc0: on cpu0 cpu1: on acpi0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 61a492006004920 device_attach: est1 attach returned 6 p4tcc1: on cpu1 orm0: at iomem 0xd0000-0xd1fff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ukbd0: on uhub2 kbd2 at ukbd0 uhid0: on uhub2 ums0: on uhub2 ums0: 8 buttons and Z dir. Timecounters tick every 1.000 msec firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me) firewire0: bus manager 0 (me) ad6: 476938MB at ata3-master SATA300 GEOM_JOURNAL: Journal 3939325718: ad6s1f contains data. GEOM_JOURNAL: Journal 3939325718: ad6s1f contains journal. GEOM_JOURNAL: Journal ad6s1f clean. acd0: DVDR at ata4-master SATA150 pcm0: pcm0: pcm1: pcm1: SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/ad6s1a cryptosoft0: on motherboard GEOM_ELI: Device ad6s1g.eli created. GEOM_ELI: Encryption: Blowfish-CBC 448 GEOM_ELI: Crypto: software GEOM_JOURNAL: Journal 2001271740: ad6s1g.eli contains data. GEOM_JOURNAL: Journal 2001271740: ad6s1g.eli contains journal. GEOM_JOURNAL: Journal ad6s1g.eli clean. GEOM_ELI: Device ad6s1b.eli created. GEOM_ELI: Encryption: AES-CBC 256 GEOM_ELI: Crypto: software 2) As you can see above, GENERIC (here stable, but also occurs on BETA2). 3) loader.conf: acpi_load="YES" acpi_video_load="YES" beastie_disable="YES" geom_journal_load="YES" #smb_load="YES" #smbus_load="YES" #ichsmb_load="YES" snd_hda_load="YES" #aio_load="YES" #kqemu_load="YES" kern.cam.scsi_delay=1000 autoboot_delay=3 linux_load="YES" linprocfs_load="YES" linsysfs_load="YES" 4) device.hints unchanged. > It's been confirmed by numerous people now, including #bsdports users, > that "USB Legacy" does not work for some individuals. This is either > because of BIOS bugs, or because the USB keyboards do not support > tying into SMM. We don't know the true cause. I'm not sure, if every BIOS has got such a setting. I'm not fully sure, if this is a BIOS bug. It could be, of course. Gigabyte has released BIOS firmware updates that are not usable, until one installs Windows (the changes history does not mention any USB fixes though). It will take some time until I can patch the firmware. > One thing we do know: we have FreeBSD users stating they cannot type > in boot0/boot2/loader, even with USB Legacy enabled, so going into > single-user after a reboot is impossible. > > Another thing we do know: we have FreeBSD users who do not have fully > functional USB keyboards in FreeBSD (some see ukbd attach, others do > not; some are using USB4BSD, others are not). Yes. These are 3 different problems. 1) No keyboard in bootloader => missing BIOS USB support. 2) No keyboard after USB controller initialisation => missing quirks? 3) No keyboard spontaneously while working => bug? > So, can someone take the time to come up with test scenarios/cases so > that users can perform these tests, list off the exact hardware they > have, and we can see if there is a consistent/common failure between > everyone? If you need anything more, I can try to deliver the information. I sometimes run out of ideas how to avoid annoying the developers. :) (In other words, I have more problems to report waiting in the queue...) ;) -- Martin From thierry.herbelot at free.fr Wed Nov 12 11:13:19 2008 From: thierry.herbelot at free.fr (Thierry Herbelot) Date: Wed Nov 12 11:50:18 2008 Subject: FreeBSD 5.4 - filesystem full In-Reply-To: <20081112145105.T65116@comanche.metrocom.ru> References: <20081112145105.T65116@comanche.metrocom.ru> Message-ID: <200811121954.57350.thierry.herbelot@free.fr> Le Wednesday 12 November 2008, Varshavchick Alexander a ?crit : > I have an old enough server with FreeBSD 5.4 which from time to time > complains about filesystem full. But the problem is that the partition > in question has about 15G free space and more than 10000000 free inodes. > Then all by itself the error dissapears, only to be repeated several hours > later. What can it be and where to look? The server runs mainly apache and > sendmail, nothing special. Hello, I saw a full disk because of a runaway background fsck : bg_fsck built some image of the disk in the top-level ".snap" directory, which grew and grew and grew .... the workaround was to reboot in single-user, then fsck in foreground, and finally switch to Zfs (but obviously, only for a Releng7 machine) TfH > > Thanks and regards > > ---- > Alexander Varshavchick, Metrocom Joint Stock Company > Phone: (812)718-3322, 718-3115(fax) > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From DarwinsKernel at gmail.com Wed Nov 12 12:37:26 2008 From: DarwinsKernel at gmail.com (Charles Darwin) Date: Wed Nov 12 12:46:16 2008 Subject: Is chflags' "nodump + sunlnk" = "uchg" Message-ID: <99926641-6491-4F76-B2D2-73B1C6D52033@gmail.com> Hi all, Title is the question actually: Is chflags' "nodump + sunlnk" = "uchg" Thanks, Charles From dnelson at allantgroup.com Wed Nov 12 15:43:49 2008 From: dnelson at allantgroup.com (Dan Nelson) Date: Wed Nov 12 15:43:56 2008 Subject: Is chflags' "nodump + sunlnk" = "uchg" In-Reply-To: <99926641-6491-4F76-B2D2-73B1C6D52033@gmail.com> References: <99926641-6491-4F76-B2D2-73B1C6D52033@gmail.com> Message-ID: <20081112234345.GL85407@dan.emsphone.com> In the last episode (Nov 12), Charles Darwin said: > Hi all, > > Title is the question actually: Is chflags' "nodump + sunlnk" = "uchg" No; why would it be? From /usr/include/sys/stat.h: #define UF_NODUMP 0x00000001 /* do not dump file */ #define SF_NOUNLINK 0x00100000 /* file may not be removed or renamed */ #define UF_IMMUTABLE 0x00000002 /* file may not be changed */ nodump+sunlnk would be 0x00100001, while uchg is 0x00000002 . # touch a b # chflags nodump,sunlnk a # chflags uchg b # ls -lo a b -rw-r--r-- 1 root wheel sunlnk,nodump 0 Nov 12 17:42 a -rw-r--r-- 1 root wheel uchg 0 Nov 12 17:42 b # -- Dan Nelson dnelson@allantgroup.com From eitanadlerlist at gmail.com Wed Nov 12 16:38:41 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Wed Nov 12 16:38:49 2008 Subject: uname -c: alias for uname -rms In-Reply-To: <200811111132.14779.jhb@freebsd.org> References: <491764C3.5020504@gmail.com> <491782BD.9090009@freebsd.org> <49179E9A.2020507@gmail.com> <200811111132.14779.jhb@freebsd.org> Message-ID: <491B770C.5060703@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 John Baldwin wrote: > On Sunday 09 November 2008 09:38:18 pm Eitan Adler wrote: >> Tim Kientzle wrote: >>> Are there any precedents for this option? >> Not really - its more like the recent cp -a switch: an alias for a >> common set of switches. > > However, cp -a was added to be compatible with other OS's, not purely as a > shortcut. > Ah. I was not aware of this. - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkbdwwACgkQtl8kq+nCzNH47QCfYxuNm0X8GkWUsh/w5d7TLose o0oAoIPDoNFNj0VIYWrITp3uPDsXrGRI =cWEU -----END PGP SIGNATURE----- From rmaglasang at infoweapons.com Thu Nov 13 02:21:58 2008 From: rmaglasang at infoweapons.com (Ronnel P. Maglasang) Date: Thu Nov 13 02:22:20 2008 Subject: assigning interrupts Message-ID: <491BFB68.7050405@infoweapons.com> Hi All, Is there a way to explicitly assign an interrupt of a device? I'm running on 6.3 and the two NICs share the same interrupt. Obviously this will affect the performance if the NICs are exposed to heavy network traffic. # vmstat -i interrupt total rate irq11: em0 vr0+ 1081099 77 Total 16958562 1222 Looking at the driver's code, I have the initial though that this is the place where I can modify. -- adapter->res_interrupt = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); -- I've tried changing RF_SHAREABLE to RF_ALLOCATED or other values but still could not get the desired result and worst the device fail to initialize. Is this possible in 6.3? Thanks, Ronnel From rea-fbsd at codelabs.ru Thu Nov 13 02:31:24 2008 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Thu Nov 13 02:31:33 2008 Subject: assigning interrupts In-Reply-To: <491BFB68.7050405@infoweapons.com> References: <491BFB68.7050405@infoweapons.com> Message-ID: Thu, Nov 13, 2008 at 06:03:20PM +0800, Ronnel P. Maglasang wrote: > Is there a way to explicitly assign an interrupt > of a device? What about BIOS? What about physically reshuffling the cards if they aren't on-board ones? -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081113/ab3fd770/attachment.pgp From koitsu at FreeBSD.org Thu Nov 13 02:40:56 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Thu Nov 13 02:41:08 2008 Subject: assigning interrupts In-Reply-To: <491BFB68.7050405@infoweapons.com> References: <491BFB68.7050405@infoweapons.com> Message-ID: <20081113104054.GA17501@icarus.home.lan> On Thu, Nov 13, 2008 at 06:03:20PM +0800, Ronnel P. Maglasang wrote: > Hi All, > > Is there a way to explicitly assign an interrupt > of a device? I'm running on 6.3 and the two NICs > share the same interrupt. Obviously this will affect > the performance if the NICs are exposed to heavy network > traffic. > > # vmstat -i > interrupt total rate > > irq11: em0 vr0+ 1081099 77 > > Total 16958562 1222 > > > Looking at the driver's code, I have the initial though > that this is the place where I can modify. This is the responsibility of the BIOS or ACPI configuration. There is no way to do this via OS software, as far as I know. Try looking at the motherboard manual for what PCI levels (A/B/C/D) share IRQs with what slot, then move cards around. Otherwise, consider purchasing a motherboard that has an APIC (this is not a typo) increasing the IRQ count to 256. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From alex at metrocom.ru Thu Nov 13 02:43:41 2008 From: alex at metrocom.ru (Varshavchick Alexander) Date: Thu Nov 13 02:43:48 2008 Subject: FreeBSD 5.4 - filesystem full In-Reply-To: References: <20081112145105.T65116@comanche.metrocom.ru> Message-ID: <20081113133157.J52884@comanche.metrocom.ru> On Wed, 12 Nov 2008, Ivan Voras wrote: > I don't know for sure, but here's some generic troubleshooting: > > a) Are you 100% sure there isn't an application that periodically fills the > drive? This would be easiest to solve - all other problems are worse. Yes, I'm sure, anyways there is about 40G free space now on that drive and it's not so easy to suddenly fill it up > b) How is your IO rate at the time you run out of space? Nothing ususial, except I just noticed that at any time, not just when the problem arises, the %slo-z value is about 97-99%. May be it has nothing to do with this problem, but on other similar servers it's not that high. > c) Did you try fsck-ing the file system? Last time two weeks ago the server was rebooted ungracefully and fsck'd in foreground (it has background_fsck="NO"). After that, I didn't have a possibility yet to reboot it in single-user. If I could do it remotely it would help but there is no KVM or anything there. Honestly I'm a bit afraid of doing "kill -1 1" or "reboot -qn". > d) Why can't you upgrade to a more recent version of FreeBSD, like 7.1? The server is in production and there is no other server to substitute it during the downtime. ---- Alexander Varshavchick, Metrocom Joint Stock Company Phone: (812)718-3322, 718-3115(fax) From alex at metrocom.ru Thu Nov 13 03:02:19 2008 From: alex at metrocom.ru (Varshavchick Alexander) Date: Thu Nov 13 03:02:29 2008 Subject: FreeBSD 5.4 - filesystem full In-Reply-To: <78cb3d3f0811120944n5e57cbf5xc1776930aae85c06@mail.gmail.com> References: <20081112145105.T65116@comanche.metrocom.ru> <78cb3d3f0811120944n5e57cbf5xc1776930aae85c06@mail.gmail.com> Message-ID: <20081113135822.L52884@comanche.metrocom.ru> On Wed, 12 Nov 2008, Adrian Penisoara wrote: > What kind of applications are you running on the machine ? Are they > mmap'ing files on the filesystem in quesiton (which one ?) ? mainly apache, sphinx's search daemon and several perl scripts > AFAIR even if you delete a big file the disk space may not be > reclaimed if a process still has the file open. but even if you run df -ki in the exact moment of when the filesystem full messages are appearing in the logs, it reports of having 40G free and a lot of free inodes. > > If you reboot the machine or restart some of the applications, does > the issue disappear ? after rebooting during several days the issue doesn't arise, then it repeats again. ---- Alexander Varshavchick, Metrocom Joint Stock Company Phone: (812)718-3322, 718-3115(fax) From darwinskernel at gmail.com Thu Nov 13 05:55:03 2008 From: darwinskernel at gmail.com (Charles Darwin) Date: Thu Nov 13 06:43:17 2008 Subject: Is chflags' "nodump + sunlnk" = "uchg" In-Reply-To: <20081112234345.GL85407@dan.emsphone.com> References: <99926641-6491-4F76-B2D2-73B1C6D52033@gmail.com> <20081112234345.GL85407@dan.emsphone.com> Message-ID: <318554EF-A7A4-41B6-8B81-6C3B9AFF4013@gmail.com> On 12-Nov-08, at 6:43 PM, Dan Nelson wrote: > In the last episode (Nov 12), Charles Darwin said: >> Hi all, >> >> Title is the question actually: Is chflags' "nodump + sunlnk" = >> "uchg" > > No; why would it be? I mean as far as their effect on a directory; doesn't "Don't change" for a directory mean "Don't add + Don't remove"? Charles From joerg at britannica.bec.de Thu Nov 13 08:47:43 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Thu Nov 13 08:47:56 2008 Subject: assigning interrupts In-Reply-To: <20081113104054.GA17501@icarus.home.lan> References: <491BFB68.7050405@infoweapons.com> <20081113104054.GA17501@icarus.home.lan> Message-ID: <20081113154003.GC1750@britannica.bec.de> On Thu, Nov 13, 2008 at 02:40:54AM -0800, Jeremy Chadwick wrote: > Otherwise, consider purchasing a motherboard that has an APIC (this is > not a typo) increasing the IRQ count to 256. This is wrong. The first IO-APIC gives you 8 additional interrupts to the 16 ISA interrupt lines. Every additional IO-APIC gives you 24 more. Most modern chipsets have one IO-APIC, at least for non-embedded systems. It doesn't mean you don't get interrupt sharing though. Joerg From koitsu at FreeBSD.org Thu Nov 13 08:56:36 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Thu Nov 13 08:56:44 2008 Subject: assigning interrupts In-Reply-To: <20081113154003.GC1750@britannica.bec.de> References: <491BFB68.7050405@infoweapons.com> <20081113104054.GA17501@icarus.home.lan> <20081113154003.GC1750@britannica.bec.de> Message-ID: <20081113165631.GA26469@icarus.home.lan> On Thu, Nov 13, 2008 at 04:40:03PM +0100, Joerg Sonnenberger wrote: > On Thu, Nov 13, 2008 at 02:40:54AM -0800, Jeremy Chadwick wrote: > > Otherwise, consider purchasing a motherboard that has an APIC (this is > > not a typo) increasing the IRQ count to 256. > > This is wrong. The first IO-APIC gives you 8 additional interrupts to > the 16 ISA interrupt lines. Every additional IO-APIC gives you 24 more. > Most modern chipsets have one IO-APIC, at least for non-embedded > systems. It doesn't mean you don't get interrupt sharing though. I think the problem is that I was thinking of local APICs, which provide a few hundred (I don't remember the exact number) IRQs to an I/O APIC. For what it's worth, the devices he listed are exclusively on the PCI bus. Regarding "it means you can still get interrupt sharing", I'd like to hear more about why/how that's possible with a system sporting at least one I/O APIC. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From charmik24 at googlemail.com Thu Nov 13 07:51:12 2008 From: charmik24 at googlemail.com (O.) Date: Thu Nov 13 09:00:01 2008 Subject: Patch for working AMD Geode CS5530 audio driver on HEAD Message-ID: <200811131629.45106.charmik24@gmail.com> Hi Alec, did you find the solution? I have the same problem. Best Regards, O. >I just got around to playing with this again, to no avail: > > **** geode Probe devid 20821022 classid 00000010! > > **** geode Probe devid 20931022 classid 00000004! > > pcm0: port 0xfe00-0xfe7f irq 11 at device 15.3 on pci0 > > **** geode Attach! > > ---> Geode mem regs at fe01 > > **** AUDIO PCI HDR *** > -->Vendor ID =1022 > -->Dev ID =2093 > -->PCI cmd =5 > -->PCI status =2a0 > -->Dev revision =1 > -->PCI class =ffffffff > -->PCI latency =0 > -->PCI header type =0 > -->BIST =0 > --> Register Base address =fe01 > pcm0: calling bus_alloc_resource > pcm0: failed to enable memory mapping! > pcm0: unable to map BAR reg > device_attach: pcm0 attach returned 6 >Anyone got any ideas? Remote access is available upon request. > >-- >Alec Kloss alec at SetFilePointer.com IM: angryspamhater at yahoo.com >PGP key at http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA241980E >"No Bunny!" -- Simon, http://wiki.adultswim.com/xwiki/bin/Frisky+Dingo/Simon From joerg at britannica.bec.de Thu Nov 13 09:39:18 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Thu Nov 13 09:39:24 2008 Subject: assigning interrupts In-Reply-To: <20081113165631.GA26469@icarus.home.lan> References: <491BFB68.7050405@infoweapons.com> <20081113104054.GA17501@icarus.home.lan> <20081113154003.GC1750@britannica.bec.de> <20081113165631.GA26469@icarus.home.lan> Message-ID: <20081113173452.GA8646@britannica.bec.de> On Thu, Nov 13, 2008 at 08:56:31AM -0800, Jeremy Chadwick wrote: > Regarding "it means you can still get interrupt sharing", I'd like to > hear more about why/how that's possible with a system sporting at least > one I/O APIC. You still have a limited number of interrupt lines. Many non-highend mainboards have 4 or 8 interrupt lines. You often have more than 8 PCI devices that want interrupts (e.g. VGA, audio, 3 USB controllers, 1 EHCI contoller and the SATA controller are enough to consume all lines). As soon as you now add a new network devices, you end up sharing PCI lines. The IO-APIC wiring is also often fixed, so it can't be controlled by software. Joerg From dnelson at allantgroup.com Thu Nov 13 11:09:28 2008 From: dnelson at allantgroup.com (Dan Nelson) Date: Thu Nov 13 11:09:35 2008 Subject: Is chflags' "nodump + sunlnk" = "uchg" In-Reply-To: <318554EF-A7A4-41B6-8B81-6C3B9AFF4013@gmail.com> References: <99926641-6491-4F76-B2D2-73B1C6D52033@gmail.com> <20081112234345.GL85407@dan.emsphone.com> <318554EF-A7A4-41B6-8B81-6C3B9AFF4013@gmail.com> Message-ID: <20081113190925.GA22629@dan.emsphone.com> In the last episode (Nov 13), Charles Darwin said: > On 12-Nov-08, at 6:43 PM, Dan Nelson wrote: > > In the last episode (Nov 12), Charles Darwin said: > >> Hi all, > >> > >> Title is the question actually: Is chflags' "nodump + sunlnk" = > >> "uchg" > > > > No; why would it be? > > I mean as far as their effect on a directory; doesn't "Don't change" > for a directory mean "Don't add + Don't remove"? Ok, that's sort of a different question. "nodump" is only checked by /sbin/dump, so we'll skip that. "sunlnk" is a root-only flag, while "uchg" can be set or reset by the file's owner as well. So let's look at "uunlnk" and "uchg": $ mkdir lnk chg $ touch lnk/file1 chg/file1 $ chflags uunlnk lnk ; chflags uchg chg $ ls -lo total 8 drwxr-xr-x 4 dan wheel - 512 Nov 13 12:59 ./ drwxrwxrwt 24 root wheel - 1024 Nov 13 12:59 ../ drwxr-xr-x 2 dan wheel uchg 512 Nov 13 12:59 chg/ drwxr-xr-x 2 dan wheel uunlnk 512 Nov 13 12:59 lnk/ $ rm chg/file1 rm: chg/file1: Operation not permitted $ rm lnk/file1 $ touch chg/file2 touch: chg/file2: Operation not permitted $ touch lnk/file2 $ rm lnk/file2 $ mv chg chg1 mv: rename chg to chg1: Operation not permitted $ mv lnk lnk1 mv: rename lnk to lnk1: Operation not permitted $ rmdir chg rmdir: chg: Operation not permitted $ rmdir lnk rmdir: lnk: Operation not permitted So as far as directories are concerned, the only difference between uchg and uunlnk is that uchg prohibits creation and deletion of files inside it, and uunlnk allows it. You can also look at /sys/ufs/ufs/ufs_vnops.c and see what code looks at which flags. -- Dan Nelson dnelson@allantgroup.com From kientzle at freebsd.org Thu Nov 13 11:30:33 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Thu Nov 13 11:30:41 2008 Subject: Is chflags' "nodump + sunlnk" = "uchg" In-Reply-To: <20081113190925.GA22629@dan.emsphone.com> References: <99926641-6491-4F76-B2D2-73B1C6D52033@gmail.com> <20081112234345.GL85407@dan.emsphone.com> <318554EF-A7A4-41B6-8B81-6C3B9AFF4013@gmail.com> <20081113190925.GA22629@dan.emsphone.com> Message-ID: <491C8028.9030303@freebsd.org> > Ok, that's sort of a different question. "nodump" is only checked by > /sbin/dump.... tar obeys the nodump flag as well if you specify the --nodump option. Hmmm... Now that I think of it, cpio should also have that option; I'll put that on my TODO list. Tim From jhb at freebsd.org Thu Nov 13 11:47:07 2008 From: jhb at freebsd.org (John Baldwin) Date: Thu Nov 13 11:47:13 2008 Subject: assigning interrupts In-Reply-To: <491BFB68.7050405@infoweapons.com> References: <491BFB68.7050405@infoweapons.com> Message-ID: <200811131441.16427.jhb@freebsd.org> On Thursday 13 November 2008 05:03:20 am Ronnel P. Maglasang wrote: > Hi All, > > Is there a way to explicitly assign an interrupt > of a device? I'm running on 6.3 and the two NICs > share the same interrupt. Obviously this will affect > the performance if the NICs are exposed to heavy network > traffic. > > # vmstat -i > interrupt total rate > > irq11: em0 vr0+ 1081099 77 > > Total 16958562 1222 > > > Looking at the driver's code, I have the initial though > that this is the place where I can modify. > > -- > adapter->res_interrupt = bus_alloc_resource_any(dev, > SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); > -- > > I've tried changing RF_SHAREABLE to RF_ALLOCATED or other > values but still could not get the desired result and worst > the device fail to initialize. Is this possible in 6.3? You can not easily assign them, no. In many cases the interrupt pins from the devices may be hardwired to a single input pin on an interrupt controller. In that case there is nothing you can do. You can read more about the gory details here: http://people.freebsd.org/~jhb/papers/bsdcan/2007/ -- John Baldwin From jhb at freebsd.org Thu Nov 13 12:12:15 2008 From: jhb at freebsd.org (John Baldwin) Date: Thu Nov 13 12:12:22 2008 Subject: ide with DMA and ram > 4GB In-Reply-To: <200811121823.14400.marc.loerner@hob.de> References: <200811121823.14400.marc.loerner@hob.de> Message-ID: <200811131502.04370.jhb@freebsd.org> On Wednesday 12 November 2008 12:23:14 pm Marc L?rner wrote: > Hello, > I just stepped over a problem with my IDE disk running in DMA-mode > and having more than 4GB of RAM. > It seems that the whole way down GEOM, ata-disk, ata-dma never is checked > whether physical address of buffer is less than 4GB an so fits in 32bits. > => when PRD is set the address is rigorously truncated to fit into 32bit, > with buffer < 4GB all is quite fine. bus_dmamap_load() in ata-dma.c should result in bounce pages being allocated and having the data copied to pages below 4GB and having those addresses passed to the callback and stored in the PRD. -- John Baldwin From jhb at freebsd.org Thu Nov 13 12:12:22 2008 From: jhb at freebsd.org (John Baldwin) Date: Thu Nov 13 12:12:35 2008 Subject: assigning interrupts In-Reply-To: <20081113154003.GC1750@britannica.bec.de> References: <491BFB68.7050405@infoweapons.com> <20081113104054.GA17501@icarus.home.lan> <20081113154003.GC1750@britannica.bec.de> Message-ID: <200811131503.53975.jhb@freebsd.org> On Thursday 13 November 2008 10:40:03 am Joerg Sonnenberger wrote: > On Thu, Nov 13, 2008 at 02:40:54AM -0800, Jeremy Chadwick wrote: > > Otherwise, consider purchasing a motherboard that has an APIC (this is > > not a typo) increasing the IRQ count to 256. > > This is wrong. The first IO-APIC gives you 8 additional interrupts to > the 16 ISA interrupt lines. Every additional IO-APIC gives you 24 more. > Most modern chipsets have one IO-APIC, at least for non-embedded > systems. It doesn't mean you don't get interrupt sharing though. I/O APICs are not hardwired to 24 pins. Early Pentium SMP systems actually only had 16 pins on their I/O APICs. I've seen I/O APICs with 32 pins, etc. There is a register in the I/O APIC that lets software know how many pins it contains. -- John Baldwin From jhb at freebsd.org Thu Nov 13 12:12:26 2008 From: jhb at freebsd.org (John Baldwin) Date: Thu Nov 13 12:12:36 2008 Subject: assigning interrupts In-Reply-To: <20081113165631.GA26469@icarus.home.lan> References: <491BFB68.7050405@infoweapons.com> <20081113154003.GC1750@britannica.bec.de> <20081113165631.GA26469@icarus.home.lan> Message-ID: <200811131509.22976.jhb@freebsd.org> On Thursday 13 November 2008 11:56:31 am Jeremy Chadwick wrote: > On Thu, Nov 13, 2008 at 04:40:03PM +0100, Joerg Sonnenberger wrote: > > On Thu, Nov 13, 2008 at 02:40:54AM -0800, Jeremy Chadwick wrote: > > > Otherwise, consider purchasing a motherboard that has an APIC (this is > > > not a typo) increasing the IRQ count to 256. > > > > This is wrong. The first IO-APIC gives you 8 additional interrupts to > > the 16 ISA interrupt lines. Every additional IO-APIC gives you 24 more. > > Most modern chipsets have one IO-APIC, at least for non-embedded > > systems. It doesn't mean you don't get interrupt sharing though. > > I think the problem is that I was thinking of local APICs, which provide > a few hundred (I don't remember the exact number) IRQs to an I/O APIC. It doesn't really work that way. While local APICs do have interrupt pins, they are used for things like NMI or interrupt requests from the 8259As. I/O APIC interrupts (and MSI interrupts) are delivered as messages to the local APIC. For older APIC systems (< PII-Xeon) there was a dedicated 3-wire bus between the local and I/O APICs on which these messages were delivered. For newer systems the APIC messages are actually passed on the main system bus. Part of the message contains the IDT vector to be triggered for this interrupt (which ranges from 0 - 255, but 0 - 31 are reserved for CPU faults and exceptions). IRQ numbers are a made up cookie system that FreeBSD uses to number interrupt sources (pins on interrupt controllers like 8259A or I/O APICs, or MSI messages on PCI devices). > Regarding "it means you can still get interrupt sharing", I'd like to > hear more about why/how that's possible with a system sporting at least > one I/O APIC. The motherboard is free to tie the interrupts pins from two different PCI devices to a single input pin on the I/O APIC. There is no hard requirement that each device gets a dedicated pin. -- John Baldwin From ivoras at freebsd.org Thu Nov 13 13:40:04 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Thu Nov 13 13:40:11 2008 Subject: 128 Bucket Failures? In-Reply-To: References: Message-ID: Chris Pratt wrote: > I have asked this before a couple of years ago but received no > replies. I assumed that's because it's a somewhat obscure question. > I'm still interested and thought I might try again in case someone > new is watching this list who might know. > > A vmstat -z on my highest traffic server always shows the failures > as below on 128 Bucket. It also goes to having 0 free rather soon > after the system is restarted and never returns to having more than > 1 free in that column and yet always has the highest number of > requests by far. Does this mean anything significant? Is it > something I should tune or even can be tuned? UMA buckets seem to be some kind of cache for SMP-optimized allocations - I hope someone who knows it better will explain them. > Here is the output of the vmstat -z with everything chopped out > besides the 128 Bucket line. The machine it's on is an 8 core 8 GB > Tyan and shouldn't really be starved for anything in my way of thinking. > > vmstat -z > ITEM SIZE LIMIT USED FREE REQUESTS FAILURES > > 128 Bucket: 1048, 0, 2043, 0, 13591, 6511069 What is the server used for? Here's a snapshot from a very loaded apache+php+pgsql web server, uptime 60 days (since the last power outage): 16 Bucket: 76, 0, 42, 58, 125, 0 32 Bucket: 140, 0, 76, 64, 183, 0 64 Bucket: 268, 0, 74, 38, 438, 11 128 Bucket: 524, 0, 2060, 642, 788828, 6985 A generic advice would be to increase vm.kmem_size (you're using AMD64, right?) and see what happens. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 258 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081113/a744b156/signature.pgp From neldredge at math.ucsd.edu Thu Nov 13 23:05:22 2008 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Thu Nov 13 23:05:29 2008 Subject: Unprivileged user can't set sticky bit on a file; why? Message-ID: Hi folks, FreeBSD doesn't allow an unprivileged user to set the sticky bit (mode S_ISTXT, octal 01000) on a file, though it does allow root to do so. nate@vulcan:/tmp$ chmod +t foo chmod: foo: Inappropriate file type or format nate@vulcan:/tmp$ su Password: vulcan# chmod +t foo vulcan# ls -l foo -rw-r--r-T 1 nate wheel 0 Nov 13 22:46 foo Why is this? I don't expect the sticky bit to actually do anything on a regular file in this day and age (I know what its historical behavior was, and what it does for directories), but I'd think it would be harmless to set it. Linux lets a user set the sticky bit, and Solaris silently masks it off. I came across this when trying to rsync some files which had the sticky bit set on the remote side. (It's the historical Unix archive from tuhs.org; the files in question are part of an unpacked V7 UNIX installation, for which the sticky bit of course had meaning. :-) ) It's annoying that this makes rsync fail; it messes up my mirroring script. sticky(8) says the bit "is ignored for regular files", which evidently isn't accurate. chmod(2) says "on UFS-based file systems (FFS, LFS) the sticky bit may only be set upon directories", which isn't right either since root is able to do it. src/sys/ufs/ufs/ufs_vnops.c has the following comment: /* * Privileged processes may set the sticky bit on non-directories, * as well as set the setgid bit on a file with a group that the * process is not a member of. Both of these are allowed in * jail(8). */ but does not explain why unprivileged process should be forbidden to set the sticky bit. -- Nate Eldredge neldredge@math.ucsd.edu From rmaglasang at infoweapons.com Fri Nov 14 00:26:46 2008 From: rmaglasang at infoweapons.com (Ronnel P. Maglasang) Date: Fri Nov 14 00:27:14 2008 Subject: assigning interrupts In-Reply-To: <200811131441.16427.jhb@freebsd.org> References: <491BFB68.7050405@infoweapons.com> <200811131441.16427.jhb@freebsd.org> Message-ID: <491D356A.70607@infoweapons.com> John Baldwin wrote: > On Thursday 13 November 2008 05:03:20 am Ronnel P. Maglasang wrote: > >> Hi All, >> >> Is there a way to explicitly assign an interrupt >> of a device? I'm running on 6.3 and the two NICs >> share the same interrupt. Obviously this will affect >> the performance if the NICs are exposed to heavy network >> traffic. >> >> # vmstat -i >> interrupt total rate >> >> irq11: em0 vr0+ 1081099 77 >> >> Total 16958562 1222 >> >> >> Looking at the driver's code, I have the initial though >> that this is the place where I can modify. >> >> -- >> adapter->res_interrupt = bus_alloc_resource_any(dev, >> SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); >> -- >> >> I've tried changing RF_SHAREABLE to RF_ALLOCATED or other >> values but still could not get the desired result and worst >> the device fail to initialize. Is this possible in 6.3? >> > > You can not easily assign them, no. In many cases the interrupt pins from the > devices may be hardwired to a single input pin on an interrupt controller. > In that case there is nothing you can do. You can read more about the gory > details here: > > http://people.freebsd.org/~jhb/papers/bsdcan/2007/ > > What was changed in 7.x in terms of assigning interrupts? I have another box running on 7.0 (2 NICs). I noticed there are no devices sharing interrupts. But if 6.x is installed on the same box (previous installation), the two NICs will share the same interrupt. I'm now looking at the drivers. I assume this is not NIC-firmware related. From c.kworr at gmail.com Fri Nov 14 00:45:13 2008 From: c.kworr at gmail.com (Volodymyr Kostyrko) Date: Fri Nov 14 00:45:20 2008 Subject: Unprivileged user can't set sticky bit on a file; why? In-Reply-To: References: Message-ID: Nate Eldredge wrote: > I came across this when trying to rsync some files which had the sticky > bit set on the remote side. (It's the historical Unix archive from > tuhs.org; the files in question are part of an unpacked V7 UNIX > installation, for which the sticky bit of course had meaning. :-) ) > It's annoying that this makes rsync fail; it messes up my mirroring script. You can ask rsync to change file attributes on the fly with the --chmod option. Just my 2c. -- Sphinx of black quartz judge my vow. From neldredge at math.ucsd.edu Fri Nov 14 01:01:47 2008 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Fri Nov 14 01:01:52 2008 Subject: Unprivileged user can't set sticky bit on a file; why? In-Reply-To: References: Message-ID: On Fri, 14 Nov 2008, Volodymyr Kostyrko wrote: > Nate Eldredge wrote: > >> I came across this when trying to rsync some files which had the sticky bit >> set on the remote side. (It's the historical Unix archive from tuhs.org; >> the files in question are part of an unpacked V7 UNIX installation, for >> which the sticky bit of course had meaning. :-) ) It's annoying that this >> makes rsync fail; it messes up my mirroring script. > > You can ask rsync to change file attributes on the fly with the --chmod > option. Just my 2c. Thanks for this hint. "--chmod=F-t" solves my problem. But I am still curious about this behavior. -- Nate Eldredge neldredge@math.ucsd.edu From dwmalone at maths.tcd.ie Fri Nov 14 01:54:49 2008 From: dwmalone at maths.tcd.ie (David Malone) Date: Fri Nov 14 01:54:55 2008 Subject: Unprivileged user can't set sticky bit on a file; why? In-Reply-To: References: Message-ID: <20081114095445.GA69339@walton.maths.tcd.ie> On Thu, Nov 13, 2008 at 11:05:21PM -0800, Nate Eldredge wrote: > since root is able to do it. src/sys/ufs/ufs/ufs_vnops.c has the > following comment: > > /* > * Privileged processes may set the sticky bit on non-directories, > * as well as set the setgid bit on a file with a group that the > * process is not a member of. Both of these are allowed in > * jail(8). > */ > > but does not explain why unprivileged process should be forbidden to set > the sticky bit. I'm guessing a little, but this check has been there since the first revision of ufs_vnops.c in FreeBSD, see: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/ufs/ufs/ufs_vnops.c?annotate=1.1 line 424. When this was imported, there was still a call to vnode_pager_uncache based on the sticky bit a few lines down. This may explain why the check was there at the time - it was to stop ordinary users giving hints to the pager system. David. From rizzo at iet.unipi.it Fri Nov 14 05:25:25 2008 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Fri Nov 14 05:25:32 2008 Subject: convert bootable freebsd iso to bootable flash image Message-ID: <20081114131217.GA62275@onelab2.iet.unipi.it> Just in case people have a similar need, or can point me to better code to do the same job: i needed to convert a bootable FreeBSD iso image into a bootable flash image, and have come up with the following code (derived from PicoBSD). The nice part is that this is all done without requiring root permissions -- the iso extraction is done with bsdtar, the file system is created using makefs, and the other patching is done with bsdlabel and dd. Now i need to find something similar to convert a bootable linux image and a bootable DOS image :) cheers luigi ----------- cut here -------------------------------- #!/bin/sh # convert a FreeBSD iso to flash image # # based on picobsd tricks. # requires makefs, bsdlabel, bsdtar, sed and dd MAKEFS=makefs MKLABEL=bsdlabel BSDTAR=tar make_freebsd_image() { # tree imagefile local tree=$1 local imagefile=$2 local boot1=${tree}/boot/boot1 local boot2=${tree}/boot/boot2 echo "convert tree $tree image $img" ${MAKEFS} -t ffs -o bsize=4096 -o fsize=512 \ -f 50 ${imagefile} ${tree} ${MKLABEL} -w -f ${imagefile} auto # write a label # copy partition c: into a: with some sed magic ${MKLABEL} -f ${imagefile} | sed -e '/ c:/{p;s/c:/a:/;}' | \ ${MKLABEL} -R -f ${imagefile} /dev/stdin # dump the primary and secondary boot (primary is 512 bytes) dd if=${boot1} of=${imagefile} conv=notrunc 2>/dev/null # XXX secondary starts after the 0x114 = dec 276 bytes of the label # so we skip 276 from the source, and 276+512=788 from dst # the old style blocks used 512 and 1024 respectively dd if=${boot2} iseek=1 ibs=276 2> /dev/null | \ dd of=${imagefile} oseek=1 obs=788 conv=notrunc 2>/dev/null } tree=$1 image=$2 if [ -f $1 ] ; then echo "Extract files from ${image}" tmp="${image}.tree" mkdir -p $tmp (cd $tmp && ${BSDTAR} xf $tree) tree=$tmp fi make_freebsd_image $tree $image [ -d "$tmp" ] && (chmod -R +w $tmp && rm -rf $tmp) #------ end of fil -------------------------------- From aryeh.friedman at gmail.com Fri Nov 14 15:14:47 2008 From: aryeh.friedman at gmail.com (Aryeh M. Friedman) Date: Fri Nov 14 15:14:54 2008 Subject: looking for something like a union file system Message-ID: <491DFF57.5080204@gmail.com> I am using a dev tool that maintains a "split" source tree for currently worked on files and those in the repo (aegis which is slightly different then how svn or cvs does it) and my the default build system assumes it is all in one tree.... thus I want someway of merge the two dirs and have a copy on write via a special command for it (i.e. if I start to edit foo.c it automatically checks it out for me).... any ideas? From won.derick at yahoo.com Sat Nov 15 05:12:52 2008 From: won.derick at yahoo.com (Won De Erick) Date: Sat Nov 15 05:41:18 2008 Subject: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 Message-ID: <557765.55617.qm@web45804.mail.sp1.yahoo.com> Hello, I tested HP DL 585 (16 CPUs, w/ built-in Broadcom NICs) running FreeBSD 7.1 Beta2 under heavy network traffic (TCP). SCENARIO A : Bombarded w/ TCP traffic: When net.isr.direct=1, PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 16 root 1 171 ki31 0K 16K RUN a 65:39 15.97% idle: cpu10 28 root 1 -32 - 0K 16K WAIT 8 12:28 5.18% swi4: clock sio 15 root 1 171 ki31 0K 16K RUN b 52:46 3.76% idle: cpu11 45 root 1 -64 - 0K 16K WAIT 7 7:29 1.17% irq17: uhci0 47 root 1 -64 - 0K 16K WAIT 6 1:11 0.10% irq16: ciss0 27 root 1 -44 - 0K 16K WAIT 0 28:52 0.00% swi1: net When net.isr.direct=0, 16 root 1 171 ki31 0K 16K CPU10 a 106:46 92.58% idle: cpu10 19 root 1 171 ki31 0K 16K CPU7 7 133:37 89.16% idle: cpu7 27 root 1 -44 - 0K 16K WAIT 0 52:20 76.37% swi1: net 25 root 1 171 ki31 0K 16K RUN 1 132:30 70.26% idle: cpu1 26 root 1 171 ki31 0K 16K CPU0 0 111:58 64.36% idle: cpu0 15 root 1 171 ki31 0K 16K CPU11 b 81:09 57.76% idle: cpu11 52 root 1 -68 - 0K 16K WAIT b 64:00 42.97% irq32: bce1 51 root 1 -68 - 0K 16K WAIT a 38:22 12.26% irq31: bce0 45 root 1 -64 - 0K 16K WAIT 7 11:31 12.06% irq17: uhci0 47 root 1 -64 - 0K 16K WAIT 6 1:54 3.66% irq16: ciss0 28 root 1 -32 - 0K 16K WAIT 8 16:01 0.00% swi4: clock sio Overall CPU utilization has significantly dropped, but I noticed that swi1 has taken CPU0 with high utilization when the net.isr.direct=0. What does this mean? SCENARIO B : Bombarded w/ more TCP traffic: Worst thing, the box has become unresponsive (can't be PINGed, inaccessible through SSH) after more traffic was added retaining net.isr.direct=0. This is due maybe to the 100% utilization on CPU0 for sw1:net (see below result, first line). bce's and swi's seem to race each other based on the result when net.isr.direct=1, swi1 . The rest of the CPUs are sitting pretty (100% Idle). Can you shed some lights on this? When net.isr.direct=0: 27 root 1 -44 - 0K 16K CPU0 0 5:45 100.00% swi1: net 11 root 1 171 ki31 0K 16K CPU15 0 0:00 100.00% idle: cpu15 13 root 1 171 ki31 0K 16K CPU13 0 0:00 100.00% idle: cpu13 17 root 1 171 ki31 0K 16K CPU9 0 0:00 100.00% idle: cpu9 18 root 1 171 ki31 0K 16K CPU8 0 0:00 100.00% idle: cpu8 21 root 1 171 ki31 0K 16K CPU5 5 146:17 99.17% idle: cpu5 22 root 1 171 ki31 0K 16K CPU4 4 146:17 99.07% idle: cpu4 14 root 1 171 ki31 0K 16K CPU12 0 0:00 99.07% idle: cpu12 16 root 1 171 ki31 0K 16K CPU10 a 109:33 98.88% idle: cpu10 15 root 1 171 ki31 0K 16K CPU11 b 86:36 93.55% idle: cpu11 52 root 1 -68 - 0K 16K WAIT b 59:42 13.87% irq32: bce1 When net.isr.direct=1, 52 root 1 -68 - 0K 16K CPU11 b 55:04 97.66% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 33:52 73.88% irq31: bce0 16 root 1 171 ki31 0K 16K RUN a 102:42 26.86% idle: cpu10 15 root 1 171 ki31 0K 16K RUN b 81:20 3.17% idle: cpu11 28 root 1 -32 - 0K 16K WAIT e 13:40 0.00% swi4: clock sio With regards to bandwidth in all scenarios above, the result is extremely low (expected is several hundred Mb/s). Why? - iface Rx Tx Total ============================================================================== bce0: 4.69 Mb/s 10.49 Mb/s 15.18 Mb/s bce1: 20.66 Mb/s 4.68 Mb/s 25.34 Mb/s lo0: 0.00 b/s 0.00 b/s 0.00 b/s ------------------------------------------------------------------------------ total: 25.35 Mb/s 15.17 Mb/s 40.52 Mb/s Thanks, Won From koitsu at FreeBSD.org Sat Nov 15 06:16:33 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Sat Nov 15 06:16:40 2008 Subject: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 In-Reply-To: <557765.55617.qm@web45804.mail.sp1.yahoo.com> References: <557765.55617.qm@web45804.mail.sp1.yahoo.com> Message-ID: <20081115141631.GA75733@icarus.home.lan> On Sat, Nov 15, 2008 at 04:59:16AM -0800, Won De Erick wrote: > Hello, > > I tested HP DL 585 (16 CPUs, w/ built-in Broadcom NICs) running FreeBSD 7.1 Beta2 under heavy network traffic (TCP). > > SCENARIO A : Bombarded w/ TCP traffic: > > When net.isr.direct=1, > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 > 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 > 16 root 1 171 ki31 0K 16K RUN a 65:39 15.97% idle: cpu10 > 28 root 1 -32 - 0K 16K WAIT 8 12:28 5.18% swi4: clock sio > 15 root 1 171 ki31 0K 16K RUN b 52:46 3.76% idle: cpu11 > 45 root 1 -64 - 0K 16K WAIT 7 7:29 1.17% irq17: uhci0 > 47 root 1 -64 - 0K 16K WAIT 6 1:11 0.10% irq16: ciss0 > 27 root 1 -44 - 0K 16K WAIT 0 28:52 0.00% swi1: net > > When net.isr.direct=0, > > 16 root 1 171 ki31 0K 16K CPU10 a 106:46 92.58% idle: cpu10 > 19 root 1 171 ki31 0K 16K CPU7 7 133:37 89.16% idle: cpu7 > 27 root 1 -44 - 0K 16K WAIT 0 52:20 76.37% swi1: net > 25 root 1 171 ki31 0K 16K RUN 1 132:30 70.26% idle: cpu1 > 26 root 1 171 ki31 0K 16K CPU0 0 111:58 64.36% idle: cpu0 > 15 root 1 171 ki31 0K 16K CPU11 b 81:09 57.76% idle: cpu11 > 52 root 1 -68 - 0K 16K WAIT b 64:00 42.97% irq32: bce1 > 51 root 1 -68 - 0K 16K WAIT a 38:22 12.26% irq31: bce0 > 45 root 1 -64 - 0K 16K WAIT 7 11:31 12.06% irq17: uhci0 > 47 root 1 -64 - 0K 16K WAIT 6 1:54 3.66% irq16: ciss0 > 28 root 1 -32 - 0K 16K WAIT 8 16:01 0.00% swi4: clock sio > > Overall CPU utilization has significantly dropped, but I noticed that swi1 has taken CPU0 with high utilization when the net.isr.direct=0. > What does this mean? > > SCENARIO B : Bombarded w/ more TCP traffic: > > Worst thing, the box has become unresponsive (can't be PINGed, inaccessible through SSH) after more traffic was added retaining net.isr.direct=0. > This is due maybe to the 100% utilization on CPU0 for sw1:net (see below result, first line). bce's and swi's seem to race each other based on the result when net.isr.direct=1, swi1 . > The rest of the CPUs are sitting pretty (100% Idle). Can you shed some lights on this? > > When net.isr.direct=0: > 27 root 1 -44 - 0K 16K CPU0 0 5:45 100.00% swi1: net > 11 root 1 171 ki31 0K 16K CPU15 0 0:00 100.00% idle: cpu15 > 13 root 1 171 ki31 0K 16K CPU13 0 0:00 100.00% idle: cpu13 > 17 root 1 171 ki31 0K 16K CPU9 0 0:00 100.00% idle: cpu9 > 18 root 1 171 ki31 0K 16K CPU8 0 0:00 100.00% idle: cpu8 > 21 root 1 171 ki31 0K 16K CPU5 5 146:17 99.17% idle: cpu5 > 22 root 1 171 ki31 0K 16K CPU4 4 146:17 99.07% idle: cpu4 > 14 root 1 171 ki31 0K 16K CPU12 0 0:00 99.07% idle: cpu12 > 16 root 1 171 ki31 0K 16K CPU10 a 109:33 98.88% idle: cpu10 > 15 root 1 171 ki31 0K 16K CPU11 b 86:36 93.55% idle: cpu11 > 52 root 1 -68 - 0K 16K WAIT b 59:42 13.87% irq32: bce1 > > When net.isr.direct=1, > 52 root 1 -68 - 0K 16K CPU11 b 55:04 97.66% irq32: bce1 > 51 root 1 -68 - 0K 16K CPU10 a 33:52 73.88% irq31: bce0 > 16 root 1 171 ki31 0K 16K RUN a 102:42 26.86% idle: cpu10 > 15 root 1 171 ki31 0K 16K RUN b 81:20 3.17% idle: cpu11 > 28 root 1 -32 - 0K 16K WAIT e 13:40 0.00% swi4: clock sio > > With regards to bandwidth in all scenarios above, the result is extremely low (expected is several hundred Mb/s). Why? > > - iface Rx Tx Total > ============================================================================== > bce0: 4.69 Mb/s 10.49 Mb/s 15.18 Mb/s > bce1: 20.66 Mb/s 4.68 Mb/s 25.34 Mb/s > lo0: 0.00 b/s 0.00 b/s 0.00 b/s > ------------------------------------------------------------------------------ > total: 25.35 Mb/s 15.17 Mb/s 40.52 Mb/s > > > Thanks, > > Won And does this behaviour change if you use some other brand of NIC? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From uspoerlein at gmail.com Sat Nov 15 06:34:12 2008 From: uspoerlein at gmail.com (Ulrich Spoerlein) Date: Sat Nov 15 06:34:19 2008 Subject: looking for something like a union file system In-Reply-To: <491DFF57.5080204@gmail.com> References: <491DFF57.5080204@gmail.com> Message-ID: <20081115143346.GB1567@roadrunner.spoerlein.net> On Fri, 14.11.2008 at 17:44:39 -0500, Aryeh M. Friedman wrote: > I am using a dev tool that maintains a "split" source tree for currently > worked on files and those in the repo (aegis which is slightly different > then how svn or cvs does it) and my the default build system assumes it > is all in one tree.... thus I want someway of merge the two dirs and > have a copy on write via a special command for it (i.e. if I start to > edit foo.c it automatically checks it out for me).... any ideas? mount_unionfs(8) or the FUSE equivalent Cheers, Ulrich Spoerlein -- It is better to remain silent and be thought a fool, than to speak, and remove all doubt. From jhb at freebsd.org Sat Nov 15 10:01:37 2008 From: jhb at freebsd.org (John Baldwin) Date: Sat Nov 15 10:01:49 2008 Subject: assigning interrupts In-Reply-To: <491D356A.70607@infoweapons.com> References: <491BFB68.7050405@infoweapons.com> <200811131441.16427.jhb@freebsd.org> <491D356A.70607@infoweapons.com> Message-ID: <200811151123.41183.jhb@freebsd.org> On Friday 14 November 2008 03:23:06 am Ronnel P. Maglasang wrote: > John Baldwin wrote: > > On Thursday 13 November 2008 05:03:20 am Ronnel P. Maglasang wrote: > > > >> Hi All, > >> > >> Is there a way to explicitly assign an interrupt > >> of a device? I'm running on 6.3 and the two NICs > >> share the same interrupt. Obviously this will affect > >> the performance if the NICs are exposed to heavy network > >> traffic. > >> > >> # vmstat -i > >> interrupt total rate > >> > >> irq11: em0 vr0+ 1081099 77 > >> > >> Total 16958562 1222 > >> > >> > >> Looking at the driver's code, I have the initial though > >> that this is the place where I can modify. > >> > >> -- > >> adapter->res_interrupt = bus_alloc_resource_any(dev, > >> SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); > >> -- > >> > >> I've tried changing RF_SHAREABLE to RF_ALLOCATED or other > >> values but still could not get the desired result and worst > >> the device fail to initialize. Is this possible in 6.3? > >> > > > > You can not easily assign them, no. In many cases the interrupt pins from the > > devices may be hardwired to a single input pin on an interrupt controller. > > In that case there is nothing you can do. You can read more about the gory > > details here: > > > > http://people.freebsd.org/~jhb/papers/bsdcan/2007/ > > > > > What was changed in 7.x in terms of assigning interrupts? I have another > box running on 7.0 (2 NICs). I noticed there are no devices sharing > interrupts. But if 6.x is installed on the same box (previous installation), > the two NICs will share the same interrupt. I'm now looking at the drivers. > I assume this is not NIC-firmware related. MSI. Newer 6.x (6.4, possibly 6.3; if not by default on 6.3 then it can be enabled on 6.3 via 'hw.pci.enable_msi=1' tunable) will do it as well. -- John Baldwin From yanegomi at gmail.com Sat Nov 15 14:32:11 2008 From: yanegomi at gmail.com (Garrett Cooper) Date: Sat Nov 15 16:23:00 2008 Subject: Request for individuals interested in reviewing test / python topics Message-ID: <1155143A-D9F9-4988-B648-F9CDC2A4080C@gmail.com> Hello Hackers and Porters, I'm currently working on a proposal to the FreeBSD foundation to use Python Nose as a testing framework for writing tests. If there are any individuals who are experienced and interested helping review and provide insight into my plans for using nose as a testing framework / backend, it would be extremely helpful. By completing this work, we could finally settle on a single regression and unit testing framework for writing future testcases, which would in turn reduce the required overhead on the release team during releases, which may reduce ports freeze periods, also reduce development time while increasing productivity within the FreeBSD community for new feature and regression testing in the of the entire system and the ports / packages we provide to the community. Thanks, -Garrett From bsd.quest at googlemail.com Sat Nov 15 19:27:49 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Sat Nov 15 19:27:55 2008 Subject: KLD loading, liking Message-ID: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> Hello, i am looking for some infos (may be papers) about how KLD linker works. After kompiling the KLD contain two important sections: % readelf -S mymod.ko | grep set [ 7] set_sysinit_set PROGBITS 00000560 000560 000004 00 A 0 0 4 [ 8] set_modmetadata_s PROGBITS 00000564 000564 000008 00 A 0 0 4 ..... sysinit_set -contain a structure with a pointer to function which will be called by loading of KLD modmetadata_set - what kind of information is there and which functions of linking/loading use it ? What exact does the macro MODULE_DEPEND ? The man page is to short, and I guess it tell no all things that the macro does. Thanks From bsd.quest at googlemail.com Sat Nov 15 19:43:44 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Sat Nov 15 19:43:50 2008 Subject: vm_map_find Message-ID: <671bb5fc0811151943m7602476fy19e0f8ee75ea2377@mail.gmail.com> Hello, my question is about vm_map_find (9) int vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, vm_offset_t *addr, vm_size_t length, boolean_t find_space, vm_prot_t prot, vm_prot_t max, int cow); Could anyone explain what exactly parameter "cow" for ? Which values and meanings ? man page dives not enough informations about it. Thanks From bsd.quest at googlemail.com Sat Nov 15 19:26:07 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Sat Nov 15 20:25:12 2008 Subject: Make files for /usr/src/sys/dev/* Message-ID: <671bb5fc0811151911oa132f75j3995e8188cbf81b2@mail.gmail.com> hello, where are the Makefiles for drivers in /usr/src/dev/* % uname -v FreeBSD 7.0-RELEASE-p5 Thanks From rnoland at FreeBSD.org Sat Nov 15 20:27:52 2008 From: rnoland at FreeBSD.org (Robert Noland) Date: Sat Nov 15 20:28:03 2008 Subject: vm_map_find In-Reply-To: <671bb5fc0811151943m7602476fy19e0f8ee75ea2377@mail.gmail.com> References: <671bb5fc0811151943m7602476fy19e0f8ee75ea2377@mail.gmail.com> Message-ID: <1226808653.1739.5.camel@wombat.2hip.net> On Sun, 2008-11-16 at 04:42 +0059, Alexej Sokolov wrote: > Hello, > my question is about vm_map_find (9) > int > vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > vm_offset_t *addr, vm_size_t length, boolean_t find_space, > vm_prot_t prot, vm_prot_t max, int cow); > > Could anyone explain what exactly parameter "cow" for ? Which values and > meanings ? Well, cow is COPY_ON_WRITE. See vm_map(9) for the list of flags. robert. > man page dives not enough informations about it. > > Thanks > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081116/fc8d19c9/attachment.pgp From bsd.quest at googlemail.com Sat Nov 15 21:15:45 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Sat Nov 15 21:15:53 2008 Subject: vm_map_find In-Reply-To: <1226808653.1739.5.camel@wombat.2hip.net> References: <671bb5fc0811151943m7602476fy19e0f8ee75ea2377@mail.gmail.com> <1226808653.1739.5.camel@wombat.2hip.net> Message-ID: <20081116051514.GA5818@debian.samsung.router> On Sat, Nov 15, 2008 at 11:10:25PM -0500, Robert Noland wrote: > On Sun, 2008-11-16 at 04:42 +0059, Alexej Sokolov wrote: > > Hello, > > my question is about vm_map_find (9) > > int > > vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > vm_offset_t *addr, vm_size_t length, boolean_t find_space, > > vm_prot_t prot, vm_prot_t max, int cow); > > > > Could anyone explain what exactly parameter "cow" for ? Which values and > > meanings ? > > Well, cow is COPY_ON_WRITE. See vm_map(9) for the list of flags. > > robert. Ok, thanx a lot, but I find it strange that the info about possible values of "cow" isn't present in man pages vm_map_insert and vm_map_find Thnks again! > > > man page dives not enough informations about it. > > > > Thanks > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From bruce at cran.org.uk Sat Nov 15 22:42:27 2008 From: bruce at cran.org.uk (Bruce Cran) Date: Sat Nov 15 22:42:34 2008 Subject: Make files for /usr/src/sys/dev/* In-Reply-To: <671bb5fc0811151911oa132f75j3995e8188cbf81b2@mail.gmail.com> References: <671bb5fc0811151911oa132f75j3995e8188cbf81b2@mail.gmail.com> Message-ID: <20081115222450.02a0f0c5@tau.draftnet> On Sun, 16 Nov 2008 04:10:31 +0059 "Alexej Sokolov" wrote: > > where are the Makefiles for drivers in /usr/src/dev/* For drivers which can be built as modules, they're in /usr/src/sys/modules/* -- Bruce Cran From ed at 80386.nl Sun Nov 16 02:09:25 2008 From: ed at 80386.nl (Ed Schouten) Date: Sun Nov 16 02:09:32 2008 Subject: KLD loading, liking In-Reply-To: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> References: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> Message-ID: <20081116100923.GU81783@hoeg.nl> * Alexej Sokolov wrote: > sysinit_set -contain a structure with a pointer to function which will be > called by loading of KLD When you place SYSINIT() lines in your code, you can request functions to be called when the code is loaded. There are also some macro's such as MTX_SYSINIT(), which is a friendly wrapper around SYSINIT() which initialises a mutex. > modmetadata_set - what kind of information is there and which functions of > linking/loading use it ? Each loadable kernel module contains a structure that contains the module name, but also a function pointer to the routine that contains the module's load/unload function. If this function returns an error, the kernel module will not be (un)loaded. Take a look at the simple kernel modules such as snp(4). > What exact does the macro MODULE_DEPEND ? The man page is to short, and I > guess it tell no all things that the macro does. MODULE_DEPEND is used to say: this kernel module also depends on another module (i.e. the USB printer module depends on the USB code). Tools like kldload can then automatically load the missing modules. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081116/7b11a771/attachment.pgp From won.derick at yahoo.com Sun Nov 16 03:18:49 2008 From: won.derick at yahoo.com (Won De Erick) Date: Sun Nov 16 03:18:57 2008 Subject: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 References: <557765.55617.qm@web45804.mail.sp1.yahoo.com> <20081115141631.GA75733@icarus.home.lan> Message-ID: <193147.59821.qm@web45803.mail.sp1.yahoo.com> > ----- Original Message ---- > From: Jeremy Chadwick > To: Won De Erick > Cc: rwatson@freebsd.org; freebsd-hackers@freebsd.org > Sent: Saturday, November 15, 2008 10:16:31 PM > Subject: Re: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 > > On Sat, Nov 15, 2008 at 04:59:16AM -0800, Won De Erick wrote: > > Hello, > > > > I tested HP DL 585 (16 CPUs, w/ built-in Broadcom NICs) running FreeBSD 7.1 Beta2 under heavy network traffic (TCP). > > > > SCENARIO A : Bombarded w/ TCP traffic: > > > > When net.isr.direct=1, > > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 > > 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 > > 16 root 1 171 ki31 0K 16K RUN a 65:39 15.97% idle: cpu10 > > 28 root 1 -32 - 0K 16K WAIT 8 12:28 5.18% swi4: clock sio > > 15 root 1 171 ki31 0K 16K RUN b 52:46 3.76% idle: cpu11 > > 45 root 1 -64 - 0K 16K WAIT 7 7:29 1.17% irq17: uhci0 > > 47 root 1 -64 - 0K 16K WAIT 6 1:11 0.10% irq16: ciss0 > > 27 root 1 -44 - 0K 16K WAIT 0 28:52 0.00% swi1: net > > > > When net.isr.direct=0, > > > > 16 root 1 171 ki31 0K 16K CPU10 a 106:46 92.58% idle: cpu10 > > 19 root 1 171 ki31 0K 16K CPU7 7 133:37 89.16% idle: cpu7 > > 27 root 1 -44 - 0K 16K WAIT 0 52:20 76.37% swi1: net > > 25 root 1 171 ki31 0K 16K RUN 1 132:30 70.26% idle: cpu1 > > 26 root 1 171 ki31 0K 16K CPU0 0 111:58 64.36% idle: cpu0 > > 15 root 1 171 ki31 0K 16K CPU11 b 81:09 57.76% idle: cpu11 > > 52 root 1 -68 - 0K 16K WAIT b 64:00 42.97% irq32: bce1 > > 51 root 1 -68 - 0K 16K WAIT a 38:22 12.26% irq31: bce0 > > 45 root 1 -64 - 0K 16K WAIT 7 11:31 12.06% irq17: uhci0 > > 47 root 1 -64 - 0K 16K WAIT 6 1:54 3.66% irq16: ciss0 > > 28 root 1 -32 - 0K 16K WAIT 8 16:01 0.00% swi4: clock sio > > > > Overall CPU utilization has significantly dropped, but I noticed that swi1 has taken CPU0 with high utilization when the net.isr.direct=0. > > What does this mean? > > > > SCENARIO B : Bombarded w/ more TCP traffic: > > > > Worst thing, the box has become unresponsive (can't be PINGed, inaccessible through SSH) after more traffic was added retaining net.isr.direct=0. > > This is due maybe to the 100% utilization on CPU0 for sw1:net (see below result, first line). bce's and swi's seem to race each other based on the result when net.isr.direct=1, swi1 . > > The rest of the CPUs are sitting pretty (100% Idle). Can you shed some lights on this? > > > > When net.isr.direct=0: > > 27 root 1 -44 - 0K 16K CPU0 0 5:45 100.00% swi1: net > > 11 root 1 171 ki31 0K 16K CPU15 0 0:00 100.00% idle: cpu15 > > 13 root 1 171 ki31 0K 16K CPU13 0 0:00 100.00% idle: cpu13 > > 17 root 1 171 ki31 0K 16K CPU9 0 0:00 100.00% idle: cpu9 > > 18 root 1 171 ki31 0K 16K CPU8 0 0:00 100.00% idle: cpu8 > > 21 root 1 171 ki31 0K 16K CPU5 5 146:17 99.17% idle: cpu5 > > 22 root 1 171 ki31 0K 16K CPU4 4 146:17 99.07% idle: cpu4 > > 14 root 1 171 ki31 0K 16K CPU12 0 0:00 99.07% idle: cpu12 > > 16 root 1 171 ki31 0K 16K CPU10 a 109:33 98.88% idle: cpu10 > > 15 root 1 171 ki31 0K 16K CPU11 b 86:36 93.55% idle: cpu11 > > 52 root 1 -68 - 0K 16K WAIT b 59:42 13.87% irq32: bce1 > > > > When net.isr.direct=1, > > 52 root 1 -68 - 0K 16K CPU11 b 55:04 97.66% irq32: bce1 > > 51 root 1 -68 - 0K 16K CPU10 a 33:52 73.88% irq31: bce0 > > 16 root 1 171 ki31 0K 16K RUN a 102:42 26.86% idle: cpu10 > > 15 root 1 171 ki31 0K 16K RUN b 81:20 3.17% idle: cpu11 > > 28 root 1 -32 - 0K 16K WAIT e 13:40 0.00% swi4: clock sio > > > > With regards to bandwidth in all scenarios above, the result is extremely low (expected is several hundred Mb/s). Why? The below result should be under scenario B above only. > > > > - iface Rx Tx Total > > ============================================================================== > > bce0: 4.69 Mb/s 10.49 Mb/s 15.18 Mb/s > > bce1: 20.66 Mb/s 4.68 Mb/s 25.34 Mb/s > > lo0: 0.00 b/s 0.00 b/s 0.00 b/s > > ------------------------------------------------------------------------------ > > total: 25.35 Mb/s 15.17 Mb/s 40.52 Mb/s > > > > > > Thanks, > > > > Won > > And does this behaviour change if you use some other brand of NIC? With Intel Pro NIC ( 82571): When net.isr.direct=1, 49 root 1 -68 - 0K 16K CPU12 c 6:50 100.00% em0 taskq 15 root 1 171 ki31 0K 16K CPU11 b 5:47 100.00% idle: cpu11 50 root 1 -68 - 0K 16K CPU13 d 6:15 86.96% em1 taskq 25 root 1 171 ki31 0K 16K CPU1 1 9:27 79.79% idle: cpu1 28 root 1 -32 - 0K 16K WAIT 1 1:33 22.75% swi4: clock sio 13 root 1 171 ki31 0K 16K RUN d 4:14 12.26% idle: cpu13 14 root 1 171 ki31 0K 16K RUN c 3:37 0.00% idle: cpu12 em0 and em1 have high CPU utilization, and with netstat, there were packet errors. # netstat -I em0 -w 1 -d input (em0) output packets errs bytes packets errs bytes colls drops 15258 3066 22748316 18468 0 4886567 0 0 15461 3096 22783724 18379 0 5350130 0 0 When net.isr.direct=0, 12 root 1 171 ki31 0K 16K CPU14 e 22:28 100.00% idle: cpu14 20 root 1 171 ki31 0K 16K CPU6 6 24:32 97.85% idle: cpu6 25 root 1 171 ki31 0K 16K RUN 1 21:51 96.97% idle: cpu1 27 root 1 -44 - 0K 16K CPU2 2 5:12 91.55% swi1: net 13 root 1 171 ki31 0K 16K CPU13 d 11:04 86.96% idle: cpu13 14 root 1 171 ki31 0K 16K CPU12 c 10:51 81.59% idle: cpu12 49 root 1 -68 - 0K 16K CPU12 c 13:48 22.17% em0 taskq 24 root 1 171 ki31 0K 16K RUN 2 19:16 12.16% idle: cpu2 50 root 1 -68 - 0K 16K - d 13:34 11.87% em1 taskq 28 root 1 -32 - 0K 16K WAIT 3 3:48 0.00% swi4: clock sio sw1:net is taking high CPU utilization this time, but without packet errors: # netstat -I em0 -w 1 -d input (em0) output packets errs bytes packets errs bytes colls drops 4275 0 5528012 24878 0 24162198 0 0 4317 0 5585954 24880 0 24066583 0 0 Is this related to the context switching in FreeBSD 7.x? I noticed that there were no significant difference in enabling and disabling net.isr.direct in FreeBSD 6.2. Also, is there any significance of enabling device polling? > > -- > | Jeremy Chadwick jdc at parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | From Alexander at Leidinger.net Sun Nov 16 00:16:40 2008 From: Alexander at Leidinger.net (Alexander Leidinger) Date: Sun Nov 16 05:37:23 2008 Subject: Request for individuals interested in reviewing test / python topics In-Reply-To: <1155143A-D9F9-4988-B648-F9CDC2A4080C@gmail.com> References: <1155143A-D9F9-4988-B648-F9CDC2A4080C@gmail.com> Message-ID: <20081116085753.10415gcgqo0etm04@webmail.leidinger.net> Quoting Garrett Cooper (from Sat, 15 Nov 2008 14:08:02 -0800): > Hello Hackers and Porters, > I'm currently working on a proposal to the FreeBSD foundation to > use Python Nose as a testing framework for writing tests. If there are Are you aware of the history of the current regression tests? If not: It started without a structure, then some work was done to move to the perl testing framework style (really only the output of the tests, and the naming conventions in the directory). This was not completed, and newer tests may not comply. The reason for chosing the perl style was, to be able to use the extensive perl tools to - automatically run all the tests - be able to compare different runs with the perl tools - be able to generate a lot of different output formats (html/text/...) There's also a wiki page about testing, which you may want to check out: http://wiki.freebsd.org/TetIntegration I don't really know python nose. I just looked at it quickly and can not see any big benefit compared to the perl test protocol outlined above (and the stuff outlined in the wiki looks even more advanced than that). Would you please elaborate where you see the benefits of it? Note that during release building perl is needed anyway to generate the index for the ports collection. I don't know if python is required currently during the release generation. Bye, Alexander. -- I would have made a good pope. -- Richard Nixon http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From yonyossef.lists at gmail.com Sun Nov 16 06:10:20 2008 From: yonyossef.lists at gmail.com (Yony Yossef) Date: Sun Nov 16 06:10:26 2008 Subject: VLAN offloads on FreeBSD 6.3 & 7.0 Message-ID: <20def4870811160610l5289267erfd7abafb9916b706@mail.gmail.com> Hi All, I'm working on an Ethernet driver for FreeBSD, currently implementing VLAN offloads. I have two problems, one is enabling TSO over the VLAN interface and the second is enabling the VLAN filtering offload. About the TSO problem, I'm currently suffering a hugh performance penalty since I have no TSO enabled over my vlan interfaces. When I create a VLAN interface it does not inherit the features of it's mother-interface, e.g. IFCAP_TSO. Can it be done on FreeBSD 6.3 / 7.0 ? Second, my NIC is capable of holding a vlan table on HW, filtering vlans on it's own, now I need to find a way to update that HW table with added/deleted VLANs in order to use that VLAN filtering offload. One way is to recieve a ioctl from the OS of it's vlan table events (add, remove). I can't find such ioctl. Second way is to have direct access from the driver to the OS vlan table. I'm not familiar with the interface though (something parallel to vlan_group_get_device on linux) or if it's possible at all, can anyone help on this one? Thanks, Yony From max at love2party.net Sun Nov 16 07:09:01 2008 From: max at love2party.net (Max Laier) Date: Sun Nov 16 07:09:08 2008 Subject: VLAN offloads on FreeBSD 6.3 & 7.0 In-Reply-To: <20def4870811160610l5289267erfd7abafb9916b706@mail.gmail.com> References: <20def4870811160610l5289267erfd7abafb9916b706@mail.gmail.com> Message-ID: <200811161608.51813.max@love2party.net> On Sunday 16 November 2008 15:10:19 Yony Yossef wrote: > Hi All, > > I'm working on an Ethernet driver for FreeBSD, currently implementing > VLAN offloads. > I have two problems, one is enabling TSO over the VLAN interface and > the second is enabling the VLAN filtering offload. > > About the TSO problem, I'm currently suffering a hugh performance > penalty since I have no TSO enabled over my vlan interfaces. > When I create a VLAN interface it does not inherit the features of > it's mother-interface, e.g. IFCAP_TSO. > Can it be done on FreeBSD 6.3 / 7.0 ? > > Second, my NIC is capable of holding a vlan table on HW, filtering > vlans on it's own, now I need to find a way to update that HW table > with added/deleted VLANs in order to use that VLAN filtering offload. > One way is to recieve a ioctl from the OS of it's vlan table events > (add, remove). I can't find such ioctl. > > Second way is to have direct access from the driver to the OS vlan > table. I'm not familiar with the interface though (something parallel > to vlan_group_get_device on linux) or if it's possible at all, can > anyone help on this one? See http://svn.freebsd.org/viewvc/base?view=revision&revision=180510 for the VLAN tag issue. Simply EVENTHANDLER_REGISTER a function in your driver to update the hw-table on config/unconfig events. I hope this helps. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From yonyossef.lists at gmail.com Sun Nov 16 07:57:10 2008 From: yonyossef.lists at gmail.com (Yony Yossef) Date: Sun Nov 16 07:57:19 2008 Subject: VLAN offloads on FreeBSD 6.3 & 7.0 In-Reply-To: <200811161608.51813.max@love2party.net> References: <20def4870811160610l5289267erfd7abafb9916b706@mail.gmail.com> <200811161608.51813.max@love2party.net> Message-ID: <20def4870811160757v76bec46bt5cb134388b7d733f@mail.gmail.com> Thanks! it looks like a solution. On Sun, Nov 16, 2008 at 5:08 PM, Max Laier wrote: > On Sunday 16 November 2008 15:10:19 Yony Yossef wrote: >> Hi All, >> >> I'm working on an Ethernet driver for FreeBSD, currently implementing >> VLAN offloads. >> I have two problems, one is enabling TSO over the VLAN interface and >> the second is enabling the VLAN filtering offload. >> >> About the TSO problem, I'm currently suffering a hugh performance >> penalty since I have no TSO enabled over my vlan interfaces. >> When I create a VLAN interface it does not inherit the features of >> it's mother-interface, e.g. IFCAP_TSO. >> Can it be done on FreeBSD 6.3 / 7.0 ? >> >> Second, my NIC is capable of holding a vlan table on HW, filtering >> vlans on it's own, now I need to find a way to update that HW table >> with added/deleted VLANs in order to use that VLAN filtering offload. >> One way is to recieve a ioctl from the OS of it's vlan table events >> (add, remove). I can't find such ioctl. >> >> Second way is to have direct access from the driver to the OS vlan >> table. I'm not familiar with the interface though (something parallel >> to vlan_group_get_device on linux) or if it's possible at all, can >> anyone help on this one? > > See http://svn.freebsd.org/viewvc/base?view=revision&revision=180510 for the > VLAN tag issue. Simply EVENTHANDLER_REGISTER a function in your driver to > update the hw-table on config/unconfig events. I hope this helps. > > -- > /"\ Best regards, | mlaier@freebsd.org > \ / Max Laier | ICQ #67774661 > X http://pf4freebsd.love2party.net/ | mlaier@EFnet > / \ ASCII Ribbon Campaign | Against HTML Mail and News > From kientzle at freebsd.org Sun Nov 16 09:37:02 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Nov 16 09:37:09 2008 Subject: convert bootable freebsd iso to bootable flash image In-Reply-To: <20081114131217.GA62275@onelab2.iet.unipi.it> References: <20081114131217.GA62275@onelab2.iet.unipi.it> Message-ID: <4920576F.9030606@freebsd.org> Luigi Rizzo wrote: > Just in case people have a similar need, or can point me to better > code to do the same job: > > i needed to convert a bootable FreeBSD iso image into a bootable > flash image, and have come up with the following code (derived > from PicoBSD). The nice part is that this is all done without > requiring root permissions -- the iso extraction is done with > bsdtar, the file system is created using makefs, and the > other patching is done with bsdlabel and dd. It doesn't look like this would preserve file ownership, which can be important. It might be interesting to use libarchive within makefs so that makefs can generate the filesystem image directly from tar or iso archives and preserve all of the ownership, permissions, etc. Tim From yanegomi at gmail.com Sun Nov 16 09:00:31 2008 From: yanegomi at gmail.com (Garrett Cooper) Date: Sun Nov 16 09:41:34 2008 Subject: Request for individuals interested in reviewing test / python topics In-Reply-To: <20081116085753.10415gcgqo0etm04@webmail.leidinger.net> References: <1155143A-D9F9-4988-B648-F9CDC2A4080C@gmail.com> <20081116085753.10415gcgqo0etm04@webmail.leidinger.net> Message-ID: <364299f40811160900r420cd841xb8b1546692158b33@mail.gmail.com> On Sat, Nov 15, 2008 at 11:57 PM, Alexander Leidinger wrote: > Quoting Garrett Cooper (from Sat, 15 Nov 2008 14:08:02 > -0800): > >> Hello Hackers and Porters, >> I'm currently working on a proposal to the FreeBSD foundation to >> use Python Nose as a testing framework for writing tests. If there are > > Are you aware of the history of the current regression tests? > > If not: > > It started without a structure, then some work was done to move to the perl > testing framework style (really only the output of the tests, and the naming > conventions in the directory). This was not completed, and newer tests may > not comply. > > The reason for choosing the perl style was, to be able to use the extensive > perl tools to > - automatically run all the tests > - be able to compare different runs with the perl tools > - be able to generate a lot of different output formats (html/text/...) > > There's also a wiki page about testing, which you may want to check out: > http://wiki.freebsd.org/TetIntegration > > I don't really know python nose. I just looked at it quickly and can not see > any big benefit compared to the perl test protocol outlined above (and the > stuff outlined in the wiki looks even more advanced than that). Would you > please elaborate where you see the benefits of it? > > Note that during release building perl is needed anyway to generate the > index for the ports collection. I don't know if python is required currently > during the release generation. > > Bye, > Alexander. Alexander, Thanks for getting back with me so quickly. Let me help explain a bit... About selecting tests -- Ok. I gave a quick look at Test::Simple, Test::Harness, etc and although Perl's Test:: modules appear to be rather expedited for executing entire tests, it's not helpful when executing selected tests, unless you write wrappers -- which is part of the point where the pain in adding infrastructure comes in. The default selector plugin that comes with nose is smart enough to crack open the files in a directory structure, do some regexed based searching for ^\s+def\s+test, or something along those lines, and from that you can select individual files with testcases, individual functions in files to execute, or classes, or instances of classes, etc. So if you're developing a fix for a feature bugfix and were provided a single regression test to use for finding the issue and were adding that test to a master test file (for suite execution), but instead wanted to execute only one testcase -- nose supports that functionality out of the box with little change for the user with the default selector plugin: nosetests some_file.py:some_func Furthermore, About capture and printout -- Also, nose with plugins gives you the ability to print not just in plaintext output on a console, but also HTML (I tried looking for the example on the site that was available before, but it appears to be missing now). The example probably consisted of approximately 30~40 lines real code, which for the most part had embedded HTML formatting tags, etc. So, a basic output capture plugins with nose that prints out HTML can be written with <= 25 lines. I'm not sure whether or not Python is required for release generation, but it will have to be listed as one of the required tools in the proposal I'm going to write. Thankfully the required dependencies for Python are small (just a base system with the typical components), and if using Python v2.5+, you also have access to ctypes (a means for testing C-API's in libraries), which is incredibly beneficial I've discovered through my work at Cisco. Assembler instructions would potentially need to be added for architectures other than x86. I'll have to look into this further with Python 2.5 (we currently use Python 2.4.5 -- the last maintenance release of Python -- in our group). Hope that helps explain things a bit more, -Garrett From yanegomi at gmail.com Sun Nov 16 09:04:48 2008 From: yanegomi at gmail.com (Garrett Cooper) Date: Sun Nov 16 10:58:55 2008 Subject: Request for individuals interested in reviewing test / python topics In-Reply-To: <364299f40811160900r420cd841xb8b1546692158b33@mail.gmail.com> References: <1155143A-D9F9-4988-B648-F9CDC2A4080C@gmail.com> <20081116085753.10415gcgqo0etm04@webmail.leidinger.net> <364299f40811160900r420cd841xb8b1546692158b33@mail.gmail.com> Message-ID: <364299f40811160904n33e92a02t6cbeee414bf8c2bf@mail.gmail.com> On Sun, Nov 16, 2008 at 9:00 AM, Garrett Cooper wrote: > On Sat, Nov 15, 2008 at 11:57 PM, Alexander Leidinger > wrote: >> Quoting Garrett Cooper (from Sat, 15 Nov 2008 14:08:02 >> -0800): >> >>> Hello Hackers and Porters, >>> I'm currently working on a proposal to the FreeBSD foundation to >>> use Python Nose as a testing framework for writing tests. If there are >> >> Are you aware of the history of the current regression tests? >> >> If not: >> >> It started without a structure, then some work was done to move to the perl >> testing framework style (really only the output of the tests, and the naming >> conventions in the directory). This was not completed, and newer tests may >> not comply. >> >> The reason for choosing the perl style was, to be able to use the extensive >> perl tools to >> - automatically run all the tests >> - be able to compare different runs with the perl tools >> - be able to generate a lot of different output formats (html/text/...) >> >> There's also a wiki page about testing, which you may want to check out: >> http://wiki.freebsd.org/TetIntegration >> >> I don't really know python nose. I just looked at it quickly and can not see >> any big benefit compared to the perl test protocol outlined above (and the >> stuff outlined in the wiki looks even more advanced than that). Would you >> please elaborate where you see the benefits of it? >> >> Note that during release building perl is needed anyway to generate the >> index for the ports collection. I don't know if python is required currently >> during the release generation. >> >> Bye, >> Alexander. > > Alexander, Wanted to clarify a few points where I didn't properly word things: > Thanks for getting back with me so quickly. Let me help explain a bit... [..] > executing entire tests, it's not helpful when executing selected s/entire tests/test suites/ [..] > developing a fix for a feature bugfix and were provided a single s/feature bugfix/feature/ [..] > Furthermore, s/Furthermore,// Thanks, -Garrett From psteele at maxiscale.com Sun Nov 16 15:11:19 2008 From: psteele at maxiscale.com (Peter Steele) Date: Sun Nov 16 15:11:25 2008 Subject: How can I add new binaries to the mfsroot image? Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> I want to make a custom FreeBSD install CD-ROM with additional commands available in the mfsroot image. Adding the new commands to the image is easy enough, and I've made an install.cfg file on the CD-ROM as well so that when the CD runs the commands in install.cfg are automatically executed. This all works, except none of the new binaries I add to the mfsroot image run during the automated sysinstall session. If I reference one of the default commands (the ones stored in /stand) they run fine, but if I add a new FreeBSD binary to the /stand directory (e.g. gmirror), the command fails. What's weird is that I can open a fixit shell after the install.cfg script fails and then run the same commands interactively and they work fine. Why would work these commands work in an interactive fixit shell but not during the automated sysinstall session? From neldredge at math.ucsd.edu Sun Nov 16 15:41:07 2008 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Sun Nov 16 15:41:27 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> Message-ID: On Sun, 16 Nov 2008, Peter Steele wrote: > I want to make a custom FreeBSD install CD-ROM with additional commands > available in the mfsroot image. Adding the new commands to the image is > easy enough, and I've made an install.cfg file on the CD-ROM as well so > that when the CD runs the commands in install.cfg are automatically > executed. This all works, except none of the new binaries I add to the > mfsroot image run during the automated sysinstall session. If I > reference one of the default commands (the ones stored in /stand) they > run fine, but if I add a new FreeBSD binary to the /stand directory > (e.g. gmirror), the command fails. How does it fail? Is the binary you added statically linked? > What's weird is that I can open a fixit shell after the install.cfg > script fails and then run the same commands interactively and they work > fine. Why would work these commands work in an interactive fixit shell > but not during the automated sysinstall session? Wild guess: the shared libraries are present somewhere else on the CD, which perhaps is either not mounted or not pointed to by LD_LIBRARY_PATH or similar until the fixit shell is run. -- Nate Eldredge neldredge@math.ucsd.edu From naddy at mips.inka.de Sun Nov 16 15:41:46 2008 From: naddy at mips.inka.de (Christian Weisgerber) Date: Sun Nov 16 15:41:53 2008 Subject: VLAN offloads on FreeBSD 6.3 & 7.0 References: <20def4870811160610l5289267erfd7abafb9916b706@mail.gmail.com> Message-ID: Yony Yossef wrote: > Second, my NIC is capable of holding a vlan table on HW, filtering > vlans on it's own, now I need to find a way to update that HW table > with added/deleted VLANs in order to use that VLAN filtering offload. If I may piggyback a question here: What is the use of a HW VLAN filter? In what scenario do we receive significant traffic of frames with our station address or multicast but not for any VLAN we have joined? -- Christian "naddy" Weisgerber naddy@mips.inka.de From tremblett at gmail.com Sun Nov 16 17:04:25 2008 From: tremblett at gmail.com (Steve Tremblett) Date: Sun Nov 16 17:04:31 2008 Subject: Intel 5100 WiFi Message-ID: Is there any development being done to support the Intel 5100 wireless device? Linux support is very recent (within the last month), and I believe it requires a firmware blob. Does FreeBSD align with OpenBSD on the whole blob debate? From toyj at union.edu Sun Nov 16 18:19:52 2008 From: toyj at union.edu (jT) Date: Sun Nov 16 18:19:58 2008 Subject: Intel 5100 WiFi In-Reply-To: References: Message-ID: <9f8af95f0811161819i58e3f8bep851ca9ae25c2a43f@mail.gmail.com> Steve, A few Iwn drivers *are* supported, mine being iwn 4965 -- is supported and has been committed to 8.0-CURRENT. You can find more information about development on this hardware here : http://www.clearchain.com/blog/posts/iwn. I'm pretty sure that there is not that much work done on your model yet. Yes it will require a firmware blob as the 4965 one does. You are required to take notice to the legal aspects of the blob via setting a key in loader.conf. In terms of FreeBSD aligning with OpenBSD, I have no idea -- i have only run BSD. We *always* prefer Free as in Freedom -- hence FreeBSD -- again not sure what the OBSD policies are. In addition this is probably a better question for freebsd-mobile or freebsd-questions Thanks and good luck -- /jT http://git.zen-sources.org/?p=kernel/zenmm.git;a=summary From psteele at maxiscale.com Sun Nov 16 19:17:42 2008 From: psteele at maxiscale.com (Peter Steele) Date: Sun Nov 16 19:17:48 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F23B5040@polaris.maxiscale.com> > How does it fail? There doesn't seem to be any error generated. Or at least I tried to capture stderr and got nothing. > Is the binary you added statically linked? The command I'm doing most of my testing with is gmirror. I pulled it from one of our operation FreeBSD boxes, and it appears to be referencing several shared libraries: # strings /stand/gmirror | grep '.so.' /libexec/ld-elf.so.1 libgeom.so.4 libsbuf.so.4 libbsdxml.so.3 libutil.so.7 libc.so.7 >Wild guess: the shared libraries are present somewhere else on the CD, >which perhaps is either not mounted or not pointed to by LD_LIBRARY_PATH >or similar until the fixit shell is run. All of these shared libraries exist under /dist, which is mounted as the FreeBSD CD. The first one is an absolute path that is in fact a symbolic link in the fixit shell that ends up pointing to a location under /dist. LD_LIBRARY_PATH is not set in the fixit shell, so I'm curious how these shared libraries are being located under /dist (the ones without the explicit path). I think you are right though, it might be related to the shared libraries. I'll try setting LD_LIBRARY_PATH explicitly to see if that solves the problem. From sam at freebsd.org Sun Nov 16 19:45:03 2008 From: sam at freebsd.org (Sam Leffler) Date: Sun Nov 16 19:45:10 2008 Subject: Intel 5100 WiFi In-Reply-To: <9f8af95f0811161819i58e3f8bep851ca9ae25c2a43f@mail.gmail.com> References: <9f8af95f0811161819i58e3f8bep851ca9ae25c2a43f@mail.gmail.com> Message-ID: <4920E8BE.7080208@freebsd.org> jT wrote: > Steve, > A few Iwn drivers *are* supported, mine being iwn 4965 -- is > supported and has been committed to 8.0-CURRENT. You can find more > information about development on this hardware here : > http://www.clearchain.com/blog/posts/iwn. I'm pretty sure that there > is not that much work done on your model yet. > > Yes it will require a firmware blob as the 4965 one does. You are > required to take notice to the legal aspects of the blob via setting a > key in loader.conf. In terms of FreeBSD aligning with OpenBSD, I have > no idea -- i have only run BSD. We *always* prefer Free as in Freedom > -- hence FreeBSD -- again not sure what the OBSD policies are. > > In addition this is probably a better question for freebsd-mobile or > freebsd-questions > > iwn firmware does not require a sysctl ack. Sam From toyj at union.edu Sun Nov 16 20:27:51 2008 From: toyj at union.edu (jT) Date: Sun Nov 16 20:27:58 2008 Subject: Intel 5100 WiFi In-Reply-To: <4920E8BE.7080208@freebsd.org> References: <9f8af95f0811161819i58e3f8bep851ca9ae25c2a43f@mail.gmail.com> <4920E8BE.7080208@freebsd.org> Message-ID: <9f8af95f0811162027g41c4460fq89cf106ad64cf505@mail.gmail.com> Sam, I know you are busy, but can you explain what about my response was wrong when you say doesn't require sysctl ack -- you mean acknowledgment? I just would like to know for my future reference and Steve sorry for my misinformation. > > iwn firmware does not require a sysctl ack. > > Sam > > -- /jT http://git.zen-sources.org/?p=kernel/zenmm.git;a=summary From max at love2party.net Sun Nov 16 20:43:27 2008 From: max at love2party.net (Max Laier) Date: Sun Nov 16 20:43:34 2008 Subject: Intel 5100 WiFi In-Reply-To: <9f8af95f0811162027g41c4460fq89cf106ad64cf505@mail.gmail.com> References: <4920E8BE.7080208@freebsd.org> <9f8af95f0811162027g41c4460fq89cf106ad64cf505@mail.gmail.com> Message-ID: <200811170543.19218.max@love2party.net> On Monday 17 November 2008 05:27:48 jT wrote: > I know you are busy, but can you explain what about my response was > wrong when you say doesn't require sysctl ack -- you mean > acknowledgment? I just would like to know for my future reference and > Steve sorry for my misinformation. the firmware for the iwn(4) powered cards is (in contrast to ipw(4)) provided by Intel under a permissive license that does not require end-user acknowledgment. It is unfortunate that Intel doesn't release all firmware similarly - so we have to live with the slight confusion stemming from that. You can identify all firmware(9) modules in the tree that require a license ack, by grepping for "FIRMWARE_LICENSE" in the module Makefiles. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From sam at freebsd.org Sun Nov 16 20:43:39 2008 From: sam at freebsd.org (Sam Leffler) Date: Sun Nov 16 20:43:45 2008 Subject: Intel 5100 WiFi In-Reply-To: <9f8af95f0811162027g41c4460fq89cf106ad64cf505@mail.gmail.com> References: <9f8af95f0811161819i58e3f8bep851ca9ae25c2a43f@mail.gmail.com> <4920E8BE.7080208@freebsd.org> <9f8af95f0811162027g41c4460fq89cf106ad64cf505@mail.gmail.com> Message-ID: <4920F67A.1010103@freebsd.org> The 4965 firmware license does not require an ack via the loader tunable. This used to be true but was changed recently as I was mistaken about the license (only iwi and ipw firmware requires the end user acknowledge the EULA). Everything else that was said seemed spot on. Sam jT wrote: > Sam, > I know you are busy, but can you explain what about my response was > wrong when you say doesn't require sysctl ack -- you mean > acknowledgment? I just would like to know for my future reference and > Steve sorry for my misinformation. > > >> iwn firmware does not require a sysctl ack. >> >> Sam >> >> >> > > > > From joerg at britannica.bec.de Sun Nov 16 20:56:11 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Sun Nov 16 20:56:18 2008 Subject: Intel 5100 WiFi In-Reply-To: <9f8af95f0811161819i58e3f8bep851ca9ae25c2a43f@mail.gmail.com> References: <9f8af95f0811161819i58e3f8bep851ca9ae25c2a43f@mail.gmail.com> Message-ID: <20081117045148.GA97@britannica.bec.de> On Sun, Nov 16, 2008 at 09:19:49PM -0500, jT wrote: > Yes it will require a firmware blob as the 4965 one does. You are > required to take notice to the legal aspects of the blob via setting a > key in loader.conf. The firmware "blob" here is simply a large binary file that is loaded directly into the card memory. Contrary to the image for first and second generation Centrino, Intel changed to license to a BSDish license, so no further user interaction is needed to accept an EULA. Note that this image is run on the card, not your CPU. Joerg From yonyossef.lists at gmail.com Sun Nov 16 22:17:16 2008 From: yonyossef.lists at gmail.com (Yony Yossef) Date: Sun Nov 16 22:17:23 2008 Subject: VLAN offloads on FreeBSD 6.3 & 7.0 In-Reply-To: <200811161608.51813.max@love2party.net> References: <20def4870811160610l5289267erfd7abafb9916b706@mail.gmail.com> <200811161608.51813.max@love2party.net> Message-ID: <20def4870811162217t7aef5cb2m5f8989c405c0a9c@mail.gmail.com> Hi Max, do you happen to know if TSO (as other capabilities) can be enabled on a VLAN interface? I've already posted this question so I'm not sending this mail to the list. On Sun, Nov 16, 2008 at 5:08 PM, Max Laier wrote: > On Sunday 16 November 2008 15:10:19 Yony Yossef wrote: >> Hi All, >> >> I'm working on an Ethernet driver for FreeBSD, currently implementing >> VLAN offloads. >> I have two problems, one is enabling TSO over the VLAN interface and >> the second is enabling the VLAN filtering offload. >> >> About the TSO problem, I'm currently suffering a hugh performance >> penalty since I have no TSO enabled over my vlan interfaces. >> When I create a VLAN interface it does not inherit the features of >> it's mother-interface, e.g. IFCAP_TSO. >> Can it be done on FreeBSD 6.3 / 7.0 ? >> >> Second, my NIC is capable of holding a vlan table on HW, filtering >> vlans on it's own, now I need to find a way to update that HW table >> with added/deleted VLANs in order to use that VLAN filtering offload. >> One way is to recieve a ioctl from the OS of it's vlan table events >> (add, remove). I can't find such ioctl. >> >> Second way is to have direct access from the driver to the OS vlan >> table. I'm not familiar with the interface though (something parallel >> to vlan_group_get_device on linux) or if it's possible at all, can >> anyone help on this one? > > See http://svn.freebsd.org/viewvc/base?view=revision&revision=180510 for the > VLAN tag issue. Simply EVENTHANDLER_REGISTER a function in your driver to > update the hw-table on config/unconfig events. I hope this helps. > > -- > /"\ Best regards, | mlaier@freebsd.org > \ / Max Laier | ICQ #67774661 > X http://pf4freebsd.love2party.net/ | mlaier@EFnet > / \ ASCII Ribbon Campaign | Against HTML Mail and News > From stas at FreeBSD.org Mon Nov 17 02:28:25 2008 From: stas at FreeBSD.org (Stanislav Sedov) Date: Mon Nov 17 02:28:32 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> Message-ID: <20081117130855.d845d507.stas@FreeBSD.org> On Sun, 16 Nov 2008 14:59:29 -0800 "Peter Steele" mentioned: > > What's weird is that I can open a fixit shell after the install.cfg > script fails and then run the same commands interactively and they work > fine. Why would work these commands work in an interactive fixit shell > but not during the automated sysinstall session? > I'm not sure, but probably the installation CD doesn't carry shared libraries at all? All binaries in /stand are static-linked ones. You could also try scripts from mfsbsd project: http://people.freebsd.org/~mm/mfsbsd/ These works for me fine for building custom installation CDs. -- Stanislav Sedov ST4096-RIPE -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081117/e3ba2760/attachment.pgp From bsd.quest at googlemail.com Mon Nov 17 03:35:53 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Mon Nov 17 03:36:00 2008 Subject: KLD loading, liking In-Reply-To: <20081116100923.GU81783@hoeg.nl> References: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> <20081116100923.GU81783@hoeg.nl> Message-ID: <20081117113522.GA2732@debian.samsung.router> On Sun, Nov 16, 2008 at 11:09:00AM +0100, Ed Schouten wrote: > * Alexej Sokolov wrote: > > What exact does the macro MODULE_DEPEND ? The man page is to short, and I > > guess it tell no all things that the macro does. > > MODULE_DEPEND is used to say: this kernel module also depends on another > module (i.e. the USB printer module depends on the USB code). Tools like > kldload can then automatically load the missing modules. Not only that. The use of the MODULE_DEPEND macro allows one module to access the variables of modules on which it depends. But man page of MODULE_DEPEND doesn't tell anything about this functionality. Hence I am looking for any good documentation of KLD loader. But I didn't find anything. May be looking in the source code is the best solution. > > -- > Ed Schouten > WWW: http://80386.nl/ -- Alexej Sokolov From won.derick at yahoo.com Mon Nov 17 01:19:36 2008 From: won.derick at yahoo.com (Won De Erick) Date: Mon Nov 17 04:27:55 2008 Subject: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 Message-ID: <245613.95922.qm@web45809.mail.sp1.yahoo.com> > ----- Original Message ---- > From: Won De Erick > To: Jeremy Chadwick > Cc: rwatson@freebsd.org; freebsd-hackers@freebsd.org > Sent: Sunday, November 16, 2008 7:18:46 PM > Subject: Re: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 > > > ----- Original Message ---- > > > From: Jeremy Chadwick > > To: Won De Erick > > Cc: rwatson@freebsd.org; freebsd-hackers@freebsd.org > > Sent: Saturday, November 15, 2008 10:16:31 PM > > Subject: Re: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 > > > > On Sat, Nov 15, 2008 at 04:59:16AM -0800, Won De Erick wrote: > > > Hello, > > > > > > I tested HP DL 585 (16 CPUs, w/ built-in Broadcom NICs) running FreeBSD 7.1 Beta2 under heavy network traffic (TCP). > > > > > > SCENARIO A : Bombarded w/ TCP traffic: > > > > > > When net.isr.direct=1, > > > > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > > 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 > > > 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 > > > 16 root 1 171 ki31 0K 16K RUN a 65:39 15.97% idle: cpu10 > > > 28 root 1 -32 - 0K 16K WAIT 8 12:28 5.18% swi4: clock sio > > > 15 root 1 171 ki31 0K 16K RUN b 52:46 3.76% idle: cpu11 > > > 45 root 1 -64 - 0K 16K WAIT 7 7:29 1.17% irq17: uhci0 > > > 47 root 1 -64 - 0K 16K WAIT 6 1:11 0.10% irq16: ciss0 > > > 27 root 1 -44 - 0K 16K WAIT 0 28:52 0.00% swi1: net > > > > > > When net.isr.direct=0, > > > > > > 16 root 1 171 ki31 0K 16K CPU10 a 106:46 92.58% idle: cpu10 > > > 19 root 1 171 ki31 0K 16K CPU7 7 133:37 89.16% idle: cpu7 > > > 27 root 1 -44 - 0K 16K WAIT 0 52:20 76.37% swi1: net > > > 25 root 1 171 ki31 0K 16K RUN 1 132:30 70.26% idle: cpu1 > > > 26 root 1 171 ki31 0K 16K CPU0 0 111:58 64.36% idle: cpu0 > > > 15 root 1 171 ki31 0K 16K CPU11 b 81:09 57.76% idle: cpu11 > > > 52 root 1 -68 - 0K 16K WAIT b 64:00 42.97% irq32: bce1 > > 51 root 1 -68 - 0K 16K WAIT a 38:22 12.26% irq31: bce0 > > > 45 root 1 -64 - 0K 16K WAIT 7 11:31 12.06% irq17: uhci0 > > > 47 root 1 -64 - 0K 16K WAIT 6 1:54 3.66% irq16: ciss0 > > > 28 root 1 -32 - 0K 16K WAIT 8 16:01 0.00% swi4: clock sio > > > > > > Overall CPU utilization has significantly dropped, but I noticed that swi1 has taken CPU0 with high utilization when the net.isr.direct=0. > > > What does this mean? > > > > > > SCENARIO B : Bombarded w/ more TCP traffic: > > > > > > Worst thing, the box has become unresponsive (can't be PINGed, inaccessible through SSH) after more traffic was added retaining net.isr.direct=0. > > > This is due maybe to the 100% utilization on CPU0 for sw1:net (see below result, first line). bce's and swi's seem to race each other based on the > result when net.isr.direct=1, swi1 . > > > The rest of the CPUs are sitting pretty (100% Idle). Can you shed some lights on this? > > > > > > When net.isr.direct=0: > > > 27 root 1 -44 - 0K 16K CPU0 0 5:45 100.00% swi1: net > > > 11 root 1 171 ki31 0K 16K CPU15 0 0:00 100.00% idle: cpu15 > > > 13 root 1 171 ki31 0K 16K CPU13 0 0:00 100.00% idle: cpu13 > > > 17 root 1 171 ki31 0K 16K CPU9 0 0:00 100.00% idle: cpu9 > > > 18 root 1 171 ki31 0K 16K CPU8 0 0:00 100.00% idle: cpu8 > > > 21 root 1 171 ki31 0K 16K CPU5 5 146:17 99.17% idle: cpu5 > > > 22 root 1 171 ki31 0K 16K CPU4 4 146:17 99.07% idle: cpu4 > > > 14 root 1 171 ki31 0K 16K CPU12 0 0:00 99.07% idle: cpu12 > > > 16 root 1 171 ki31 0K 16K CPU10 a 109:33 98.88% idle: cpu10 > > > 15 root 1 171 ki31 0K 16K CPU11 b 86:36 93.55% idle: cpu11 > > > 52 root 1 -68 - 0K 16K WAIT b 59:42 13.87% irq32: bce1 > > > > > > When net.isr.direct=1, > > > 52 root 1 -68 - 0K 16K CPU11 b 55:04 97.66% irq32: bce1 > > > 51 root 1 -68 - 0K 16K CPU10 a 33:52 73.88% irq31: bce0 > > > 16 root 1 171 ki31 0K 16K RUN a 102:42 26.86% idle: cpu10 > > > 15 root 1 171 ki31 0K 16K RUN b 81:20 3.17% idle: cpu11 > > > 28 root 1 -32 - 0K 16K WAIT e 13:40 0.00% swi4: clock sio > > > > > > With regards to bandwidth in all scenarios above, the result is extremely low (expected is several hundred Mb/s). Why? > > The below result should be under scenario B above only. > > > > > > > - iface Rx Tx Total > > > ============================================================================== > > > bce0: 4.69 Mb/s 10.49 Mb/s 15.18 Mb/s > > > bce1: 20.66 Mb/s 4.68 Mb/s 25.34 Mb/s > > > lo0: 0.00 b/s 0.00 b/s 0.00 b/s > > > ------------------------------------------------------------------------------ > > > total: 25.35 Mb/s 15.17 Mb/s 40.52 Mb/s > > > > > > > > > Thanks, > > > > > > Won > > > > And does this behaviour change if you use some other brand of NIC? > > With Intel Pro NIC ( 82571): > > When net.isr.direct=1, > > 49 root 1 -68 - 0K 16K CPU12 c 6:50 100.00% em0 taskq > 15 root 1 171 ki31 0K 16K CPU11 b 5:47 100.00% idle: cpu11 > 50 root 1 -68 - 0K 16K CPU13 d 6:15 86.96% em1 taskq > 25 root 1 171 ki31 0K 16K CPU1 1 9:27 79.79% idle: cpu1 > 28 root 1 -32 - 0K 16K WAIT 1 1:33 22.75% swi4: clock sio > 13 root 1 171 ki31 0K 16K RUN d 4:14 12.26% idle: cpu13 > 14 root 1 171 ki31 0K 16K RUN c 3:37 0.00% idle: cpu12 > > em0 and em1 have high CPU utilization, and with netstat, there were packet errors. > > # netstat -I em0 -w 1 -d > input (em0) output > packets errs bytes packets errs bytes colls drops > 15258 3066 22748316 18468 0 4886567 0 0 > 15461 3096 22783724 18379 0 5350130 0 0 > > > When net.isr.direct=0, > 12 root 1 171 ki31 0K 16K CPU14 e 22:28 100.00% idle: cpu14 > 20 root 1 171 ki31 0K 16K CPU6 6 24:32 97.85% idle: cpu6 > 25 root 1 171 ki31 0K 16K RUN 1 21:51 96.97% idle: cpu1 > 27 root 1 -44 - 0K 16K CPU2 2 5:12 91.55% swi1: net > 13 root 1 171 ki31 0K 16K CPU13 d 11:04 86.96% idle: cpu13 > 14 root 1 171 ki31 0K 16K CPU12 c 10:51 81.59% idle: cpu12 > 49 root 1 -68 - 0K 16K CPU12 c 13:48 22.17% em0 taskq > 24 root 1 171 ki31 0K 16K RUN 2 19:16 12.16% idle: cpu2 > 50 root 1 -68 - 0K 16K - d 13:34 11.87% em1 taskq > 28 root 1 -32 - 0K 16K WAIT 3 3:48 0.00% swi4: clock sio > > sw1:net is taking high CPU utilization this time, but without packet errors: > > # netstat -I em0 -w 1 -d > input (em0) output > packets errs bytes packets errs bytes colls drops > 4275 0 5528012 24878 0 24162198 0 0 > 4317 0 5585954 24880 0 24066583 0 0 > > > Is this related to the context switching in FreeBSD 7.x? I noticed that there were no significant difference in enabling and disabling net.isr.direct in FreeBSD > 6.2. > Also, is there any significance of enabling device polling? > > > > > -- > > | Jeremy Chadwick jdc at parodius.com | > > | Parodius Networking http://www.parodius.com/ | > > | UNIX Systems Administrator Mountain View, CA, USA | > > | Making life hard for others since 1977. PGP: 4BD6C0CB | > > I compiled the following em driver for Intel NIC Pro (82571) w/ FreeBSD 7.1 Beta 2 on HPDL 585 machine having 16CPUs. http://people.yandex-team.ru/~wawa/ With net.isr.direct=1, I made some changes on kthreads(default=2) for em0 and em1's rx. dev.em.0.rx_kthreads: 6 .... dev.em.1.rx_kthreads: 6 With these settings, the result is: CPU: 0.0% user, 0.0% nice, 57.2% system, 3.6% interrupt, 39.2% idle Mem: 17M Active, 7228K Inact, 156M Wired, 76K Cache, 21M Buf, 31G Free Swap: 4096M Total, 4096M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 52 root 1 43 - 0K 16K CPU12 c 41:38 100.00% em0_rx_kthread_1 51 root 1 43 - 0K 16K CPU3 3 41:38 100.00% em0_rx_kthread_0 54 root 1 -68 - 0K 16K CPU2 2 39:39 100.00% em1_txcleaner 1283 root 1 43 - 0K 16K CPU1 1 38:55 100.00% em0_rx_kthread_3 1282 root 1 43 - 0K 16K CPU10 a 38:55 100.00% em0_rx_kthread_2 1344 root 1 43 - 0K 16K CPU9 9 25:51 100.00% em0_rx_kthread_5 1343 root 1 43 - 0K 16K CPU4 4 25:51 100.00% em0_rx_kthread_4 12 root 1 171 ki31 0K 16K CPU14 e 44:28 91.70% idle: cpu14 11 root 1 171 ki31 0K 16K CPU15 f 35:18 76.86% idle: cpu15 19 root 1 171 ki31 0K 16K RUN 7 24:56 70.46% idle: cpu7 20 root 1 171 ki31 0K 16K CPU6 6 35:23 69.38% idle: cpu6 15 root 1 171 ki31 0K 16K CPU11 b 34:33 65.97% idle: cpu11 18 root 1 171 ki31 0K 16K CPU8 8 40:24 64.45% idle: cpu8 13 root 1 171 ki31 0K 16K CPU13 d 42:07 61.96% idle: cpu13 21 root 1 171 ki31 0K 16K CPU5 5 21:35 58.79% idle: cpu5 28 root 1 -32 - 0K 16K WAIT 8 33:23 57.08% swi4: clock sio 25 root 1 171 ki31 0K 16K RUN 1 18:13 50.00% idle: cpu1 1347 root 1 43 - 0K 16K WAIT 5 10:48 44.68% em1_rx_kthread_5 55 root 1 43 - 0K 16K RUN 0 18:46 43.65% em1_rx_kthread_0 56 root 1 43 - 0K 16K WAIT 6 18:50 42.97% em1_rx_kthread_1 1280 root 1 43 - 0K 16K WAIT d 16:59 41.46% em1_rx_kthread_3 1279 root 1 43 - 0K 16K WAIT 7 17:00 41.06% em1_rx_kthread_2 1346 root 1 43 - 0K 16K WAIT b 10:47 40.77% em1_rx_kthread_4 26 root 1 171 ki31 0K 16K RUN 0 19:38 10.79% idle: cpu0 50 root 1 -68 - 0K 16K WAIT f 1:41 3.86% em0_txcleaner 24 root 1 171 ki31 0K 16K RUN 2 30:28 0.00% idle: cpu2 16 root 1 171 ki31 0K 16K RUN a 29:39 0.00% idle: cpu10 17 root 1 171 ki31 0K 16K RUN 9 27:08 0.00% idle: cpu9 14 root 1 171 ki31 0K 16K RUN c 21:58 0.00% idle: cpu12 23 root 1 171 ki31 0K 16K RUN 3 11:36 0.00% idle: cpu3 22 root 1 171 ki31 0K 16K RUN 4 10:24 0.00% idle: cpu4 27 root 1 -44 - 0K 16K WAIT 2 3:04 0.00% swi1: net I am happy to see that more processors are now working, but the kthreads are consuming HIGH CPU utilizations. is there any other things that I can look into and set to minimize CPU utilization for the threads? thanks, won From des at des.no Mon Nov 17 05:34:53 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Mon Nov 17 05:35:00 2008 Subject: KLD loading, liking In-Reply-To: <20081117113522.GA2732@debian.samsung.router> (Alexej Sokolov's message of "Mon, 17 Nov 2008 12:35:22 +0100") References: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> <20081116100923.GU81783@hoeg.nl> <20081117113522.GA2732@debian.samsung.router> Message-ID: <868wri3284.fsf@ds4.des.no> Alexej Sokolov writes: > Not only that. The use of the MODULE_DEPEND macro allows one module to > access the variables of modules on which it depends. No. Any module X can access any public variable or function in the kernel or in any other module Y, but loading X will fail if Y is not already loaded. The only effect of MODULE_DEPEND is to tell the loader that Y must be loaded before X. DES -- Dag-Erling Sm?rgrav - des@des.no From murat at enderunix.org Mon Nov 17 07:42:25 2008 From: murat at enderunix.org (Murat Balaban) Date: Mon Nov 17 07:44:40 2008 Subject: Giant lock, bce and uhc using the same irq Message-ID: <1226934861.6310.25.camel@efe> Hello hackers, In one of my production servers (64-bit Intel Xeon machine) running 6.3-RELEASE-p4 (amd64) FreeBSD 6.3-RELEASE-p4 #0: Fri Sep 12 17:07:19 EEST 2008 I see this "top -S" output excerpt: 32 root 1 -68 -187 0K 16K *Giant 0 48.3H 3.08% irq17: bce1 uhci1 What I get from above is that bce and uhc drivers both use irq17. I assume bce is SMPng'ed, and uhc is not. Does uhc being Giant locked, affect bce performance, because they use the same interrupt handler thread? -- Murat http://www.enderunix.org/murat/ From free.dvig at gmail.com Mon Nov 17 08:02:04 2008 From: free.dvig at gmail.com (Aleksandr Litvinov) Date: Mon Nov 17 08:02:10 2008 Subject: =?koi8-r?b?79TXxdQ6IEtMRCBsb2FkaW5nLCBsaWtpbmc=?= In-Reply-To: <20081117113522.GA2732@debian.samsung.router> References: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> <20081116100923.GU81783@hoeg.nl> <20081117113522.GA2732@debian.samsung.router> Message-ID: <65f70ae30811170735i60891633ja0791f3d73cee188@mail.gmail.com> 2008/11/17, Alexej Sokolov : > On Sun, Nov 16, 2008 at 11:09:00AM +0100, Ed Schouten wrote: >> * Alexej Sokolov wrote: > >> > What exact does the macro MODULE_DEPEND ? The man page is to short, and >> > I >> > guess it tell no all things that the macro does. >> >> MODULE_DEPEND is used to say: this kernel module also depends on another >> module (i.e. the USB printer module depends on the USB code). Tools like >> kldload can then automatically load the missing modules. > Not only that. The use of the MODULE_DEPEND macro allows one module to > access > the variables of modules on which it depends. But man page of > MODULE_DEPEND doesn't tell anything about this functionality. Hence I > am looking for any good documentation of KLD loader. But I didn't find > anything. May be looking in the source code is the best solution. > >> >> -- >> Ed Schouten >> WWW: http://80386.nl/ > > > > -- > Alexej Sokolov > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Hello, You can receive a little information about KLD from the book "designing BSD rootkits". -- -- Good Luck. -- Litvinov Aleksandr. From kris at FreeBSD.org Mon Nov 17 09:12:50 2008 From: kris at FreeBSD.org (Kris Kennaway) Date: Mon Nov 17 09:12:57 2008 Subject: Giant lock, bce and uhc using the same irq In-Reply-To: <1226934861.6310.25.camel@efe> References: <1226934861.6310.25.camel@efe> Message-ID: <20081117171250.GA43367@hub.freebsd.org> On Mon, Nov 17, 2008 at 05:14:21PM +0200, Murat Balaban wrote: > Hello hackers, > > In one of my production servers (64-bit Intel Xeon machine) running > > 6.3-RELEASE-p4 (amd64) FreeBSD 6.3-RELEASE-p4 #0: Fri Sep 12 17:07:19 > EEST 2008 > > I see this "top -S" output excerpt: > > 32 root 1 -68 -187 0K 16K *Giant 0 48.3H 3.08% irq17: > bce1 uhci1 > > What I get from above is that bce and uhc drivers both use irq17. I > assume > bce is SMPng'ed, and uhc is not. > > Does uhc being Giant locked, affect bce performance, because they use > the same > interrupt handler thread? bce will not need to acquire Giant at all so they will not fight for the lock. However both of them will have to wake up to check each interrupt so there is some time spent there. Also if you have some other workload on the system that is still heavily dependent onGiant (e.g. MSDOSFS, etc) then that will interfere with uhci while interrupts are coming in. Kris P.S. In 8.0 there is a new USB stack that is Giant-free. -- In God we Trust -- all others must submit an X.509 certificate. -- Charles Forsythe From psteele at maxiscale.com Mon Nov 17 09:47:24 2008 From: psteele at maxiscale.com (Peter Steele) Date: Mon Nov 17 09:47:31 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <20081117130855.d845d507.stas@FreeBSD.org> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> <20081117130855.d845d507.stas@FreeBSD.org> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> >I'm not sure, but probably the installation CD doesn't carry shared libraries at all? All binaries in /stand are >static-linked ones. Yeah, that is absolutely the problem--no shared libraries are available when sysinstall is running. >You could also try scripts from mfsbsd project: >http://people.freebsd.org/~mm/mfsbsd/ > >These works for me fine for building custom installation CDs. I'll have to check this out. I'm not getting anywhere with trying to customize mfsroot with my current approach... From arhimed at gmx.net Mon Nov 17 09:52:38 2008 From: arhimed at gmx.net (Alexandre Fiveg) Date: Mon Nov 17 10:00:03 2008 Subject: KLD loading, liking In-Reply-To: <868wri3284.fsf@ds4.des.no> References: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> <20081116100923.GU81783@hoeg.nl> <20081117113522.GA2732@debian.samsung.router> <868wri3284.fsf@ds4.des.no> Message-ID: <20081117172524.GC2732@debian.samsung.router> On Mon, Nov 17, 2008 at 02:34:28PM +0100, Dag-Erling Sm?rgrav wrote: > Alexej Sokolov writes: > > Not only that. The use of the MODULE_DEPEND macro allows one module to > > access the variables of modules on which it depends. > > No. Any module X can access any public variable or function in the > kernel or in any other module Y, but loading X will fail if Y is not > already loaded. The only effect of MODULE_DEPEND is to tell the loader > that Y must be loaded before X. No, Example: two modules kld.c and kld1.1: kld: http://pastebin.com/m67799565 Makefile: http://pastebin.com/m5418e5a7 kld1: http://pastebin.com/d154e8474 Makefile: http://pastebin.com/m79723138 In kld is public int var_from_kld declared. To access this variable from kld1 you have to uncomment macro MODULE_DEPEND My system: % uname -v FreeBSD 7.0-RELEASE-p5 #0: Tue Oct 7 19:05:20 CEST 2008 Tell me please if I do something wrong! Thanx > > DES > -- > Dag-Erling Sm?rgrav - des@des.no > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Alexandre Fiveg Key fingerprint = 0B23 EB52 3944 E440 CFF3 C1F1 7D05 8D00 34F7 A6BD From ed at 80386.nl Mon Nov 17 12:44:09 2008 From: ed at 80386.nl (Ed Schouten) Date: Mon Nov 17 12:44:15 2008 Subject: =?utf-8?b?0J7RgtCy0LXRgjo=?= KLD loading, liking In-Reply-To: <65f70ae30811170735i60891633ja0791f3d73cee188@mail.gmail.com> References: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> <20081116100923.GU81783@hoeg.nl> <20081117113522.GA2732@debian.samsung.router> <65f70ae30811170735i60891633ja0791f3d73cee188@mail.gmail.com> Message-ID: <20081117204407.GY81783@hoeg.nl> * Aleksandr Litvinov wrote: > Hello, > You can receive a little information about KLD from the book > "designing BSD rootkits". I don't own this book myself, but a colleague at Snow B.V. once showed it to me. I only looked through it a couple of minutes, but it seemed like a book nice to have. It also shows some techniques on how to hide KLD's. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081117/87cab8e4/attachment.pgp From ed at 80386.nl Mon Nov 17 12:48:03 2008 From: ed at 80386.nl (Ed Schouten) Date: Mon Nov 17 12:48:09 2008 Subject: Giant lock, bce and uhc using the same irq In-Reply-To: <20081117171250.GA43367@hub.freebsd.org> References: <1226934861.6310.25.camel@efe> <20081117171250.GA43367@hub.freebsd.org> Message-ID: <20081117204802.GZ81783@hoeg.nl> * Kris Kennaway wrote: > P.S. In 8.0 there is a new USB stack that is Giant-free. I'm not sure this is completely true. Maybe HPS could explain it in more detail, but Giant still seems to be used pretty often. -- Ed Schouten WWW: http://80386.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081117/48ffe9e2/attachment.pgp From hselasky at c2i.net Mon Nov 17 13:21:43 2008 From: hselasky at c2i.net (Hans Petter Selasky) Date: Mon Nov 17 13:21:50 2008 Subject: Giant lock, bce and uhc using the same irq In-Reply-To: <20081117204802.GZ81783@hoeg.nl> References: <1226934861.6310.25.camel@efe> <20081117171250.GA43367@hub.freebsd.org> <20081117204802.GZ81783@hoeg.nl> Message-ID: <200811172223.51209.hselasky@c2i.net> On Monday 17 November 2008, Ed Schouten wrote: > * Kris Kennaway wrote: > > P.S. In 8.0 there is a new USB stack that is Giant-free. > > I'm not sure this is completely true. Maybe HPS could explain it in more > detail, but Giant still seems to be used pretty often. All the interrupt handlers of the Host- and Device-controller drivers are free from locking Giant when executing. --HPS From ivoras at freebsd.org Mon Nov 17 13:25:04 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Mon Nov 17 13:25:15 2008 Subject: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 In-Reply-To: <245613.95922.qm@web45809.mail.sp1.yahoo.com> References: <245613.95922.qm@web45809.mail.sp1.yahoo.com> Message-ID: Won De Erick wrote: > I compiled the following em driver for Intel NIC Pro (82571) w/ FreeBSD 7.1 Beta 2 on HPDL 585 machine having 16CPUs. > > http://people.yandex-team.ru/~wawa/ > > With net.isr.direct=1, I made some changes on kthreads(default=2) for em0 and em1's rx. > > dev.em.0.rx_kthreads: 6 > .... > dev.em.1.rx_kthreads: 6 > > With these settings, the result is: > > CPU: 0.0% user, 0.0% nice, 57.2% system, 3.6% interrupt, 39.2% idle > Mem: 17M Active, 7228K Inact, 156M Wired, 76K Cache, 21M Buf, 31G Free > Swap: 4096M Total, 4096M Free > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 52 root 1 43 - 0K 16K CPU12 c 41:38 100.00% em0_rx_kthread_1 > 51 root 1 43 - 0K 16K CPU3 3 41:38 100.00% em0_rx_kthread_0 > 54 root 1 -68 - 0K 16K CPU2 2 39:39 100.00% em1_txcleaner > 1283 root 1 43 - 0K 16K CPU1 1 38:55 100.00% em0_rx_kthread_3 > 1282 root 1 43 - 0K 16K CPU10 a 38:55 100.00% em0_rx_kthread_2 > 1344 root 1 43 - 0K 16K CPU9 9 25:51 100.00% em0_rx_kthread_5 > 1343 root 1 43 - 0K 16K CPU4 4 25:51 100.00% em0_rx_kthread_4 > 12 root 1 171 ki31 0K 16K CPU14 e 44:28 91.70% idle: cpu14 This is very interesting. Do you see real performance (network throughput) increase? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 258 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081117/b4a2447e/signature.pgp From psteele at maxiscale.com Mon Nov 17 15:56:54 2008 From: psteele at maxiscale.com (Peter Steele) Date: Mon Nov 17 15:57:00 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com><20081117130855.d845d507.stas@FreeBSD.org> <2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> >I'll have to check this out. I'm not getting anywhere with trying to >customize mfsroot with my current approach... The goal we are trying to achieve btw is to make gmirror available during an install so that the file systems are mirrored right from the get-go, so that we can avoid having to go through the process of converting a system as a post operation. The standard slicing/partition commands of sysinstall do support the creation of a mirrored file system though, so our idea was to run a script via install.cfg to take care of fdisk/bsdlabel/gmirror phase, and then install the packages in the normal fashion via subsequent steps in install.cfg. Is this something that can be done via sysinstall? If not, what's the best alternative? This whole process is targeted to be on a PXE boot server so we can configure our systems in a completely automated hands-off manner. We have 200+ FreeBSD systems and we definitely need an automated process. We already have it working fine, but without mirroring. We can upgrade doezens of systems at a time simply by making them boot from our PXE server. We now need to tweak this process so that we can establish the mirrored file systems as part of the automated install. From psteele at maxiscale.com Mon Nov 17 17:45:17 2008 From: psteele at maxiscale.com (Peter Steele) Date: Mon Nov 17 17:45:23 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com><20081117130855.d845d507.stas@FreeBSD.org><2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240C8F6@polaris.maxiscale.com> >I'll have to check this out. I'm not getting anywhere with trying to >customize mfsroot with my current approach... The goal we are trying to achieve btw is to make gmirror available during an install so that the file systems are mirrored right from the get-go, so that we can avoid having to go through the process of converting a system as a post operation. The standard slicing/partition commands of sysinstall do *not* support the creation of a mirrored file system though, so our idea was to run a script via install.cfg to take care of fdisk/bsdlabel/gmirror phase, and then install the packages in the normal fashion via subsequent steps in install.cfg. Is this something that can be done via sysinstall? If not, what's the best alternative? This whole process is targeted to be on a PXE boot server so we can configure our systems in a completely automated hands-off manner. We have 200+ FreeBSD systems and we definitely need an automated process. We already have it working fine, but without mirroring. We can upgrade doezens of systems at a time simply by making them boot from our PXE server. We now need to tweak this process so that we can establish the mirrored file systems as part of the automated install. From won.derick at yahoo.com Mon Nov 17 16:45:43 2008 From: won.derick at yahoo.com (Won De Erick) Date: Mon Nov 17 18:49:52 2008 Subject: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 References: <245613.95922.qm@web45809.mail.sp1.yahoo.com> Message-ID: <718671.27101.qm@web45805.mail.sp1.yahoo.com> > ----- Original Message ---- > From: Ivan Voras > To: freebsd-hackers@freebsd.org > Sent: Tuesday, November 18, 2008 5:21:04 AM > Subject: Re: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 > > Won De Erick wrote: > > > I compiled the following em driver for Intel NIC Pro (82571) w/ FreeBSD 7.1 Beta 2 on HPDL 585 machine having 16CPUs. > > > > http://people.yandex-team.ru/~wawa/ > > > > With net.isr.direct=1, I made some changes on kthreads(default=2) for em0 and em1's rx. > > > > dev.em.0.rx_kthreads: 6 > > .... > > dev.em.1.rx_kthreads: 6 > > > > With these settings, the result is: > > > > CPU: 0.0% user, 0.0% nice, 57.2% system, 3.6% interrupt, 39.2% idle > > Mem: 17M Active, 7228K Inact, 156M Wired, 76K Cache, 21M Buf, 31G Free > > Swap: 4096M Total, 4096M Free > > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > 52 root 1 43 - 0K 16K CPU12 c 41:38 100.00% em0_rx_kthread_1 > > 51 root 1 43 - 0K 16K CPU3 3 41:38 100.00% em0_rx_kthread_0 > > 54 root 1 -68 - 0K 16K CPU2 2 39:39 100.00% em1_txcleaner > > 1283 root 1 43 - 0K 16K CPU1 1 38:55 100.00% em0_rx_kthread_3 > > 1282 root 1 43 - 0K 16K CPU10 a 38:55 100.00% em0_rx_kthread_2 > > 1344 root 1 43 - 0K 16K CPU9 9 25:51 100.00% em0_rx_kthread_5 > > 1343 root 1 43 - 0K 16K CPU4 4 25:51 100.00% em0_rx_kthread_4 > > 12 root 1 171 ki31 0K 16K CPU14 e 44:28 91.70% idle: cpu14 > > This is very interesting. Do you see real performance (network > throughput) increase? > There's a little improvement on the throughput, but packet errors occurred on both interfaces. # netstat -I em1 -w 1 -d input (em1) output packets errs bytes packets errs bytes colls drops 32494 483 23083087 15681 0 23719154 0 82 30547 330 23104447 16062 0 23077442 0 44 # netstat -I em0 -w 1 -d input (em0) output packets errs bytes packets errs bytes colls drops 19889 640 24144754 21307 0 8719922 0 0 18071 2436 25966238 21088 0 8766995 0 0 Is there any other thing that I can tweak to solve the problem? From doconnor at gsoft.com.au Tue Nov 18 02:41:08 2008 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Tue Nov 18 02:41:19 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> Message-ID: <200811182111.02367.doconnor@gsoft.com.au> On Tuesday 18 November 2008 10:26:26 Peter Steele wrote: > Is this something that can be done via sysinstall? If not, what's the > best alternative? This whole process is targeted to be on a PXE boot > server so we can configure our systems in a completely automated > hands-off manner. We have 200+ FreeBSD systems and we definitely need an > automated process. We already have it working fine, but without > mirroring. We can upgrade doezens of systems at a time simply by making > them boot from our PXE server. We now need to tweak this process so that > we can establish the mirrored file systems as part of the automated > install. I believe you modify /usr/src/release/${ARCH}/boot_crunch.conf to do this. I haven't actually tried though... I think it would be possible to have a 'GEOM' menu that you can run prior to fdisk, label, etc that would allow you to do some basic stuff like this. While the sysinstall code is a bit fugly it's not that difficult to hack on (speaking from limited experience :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081118/e5fa6003/attachment.pgp From mail25 at bzerk.org Tue Nov 18 00:57:46 2008 From: mail25 at bzerk.org (Ruben de Groot) Date: Tue Nov 18 04:42:26 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> Message-ID: <20081118085739.GA95079@ei.bzerk.org> On Mon, Nov 17, 2008 at 03:56:26PM -0800, Peter Steele typed: > >I'll have to check this out. I'm not getting anywhere with trying to > >customize mfsroot with my current approach... > > The goal we are trying to achieve btw is to make gmirror available > during an install so that the file systems are mirrored right from the > get-go, so that we can avoid having to go through the process of > converting a system as a post operation. The standard slicing/partition > commands of sysinstall do support the creation of a mirrored file system > though, so our idea was to run a script via install.cfg to take care of > fdisk/bsdlabel/gmirror phase, and then install the packages in the > normal fashion via subsequent steps in install.cfg. > > Is this something that can be done via sysinstall? If not, what's the > best alternative? This whole process is targeted to be on a PXE boot > server so we can configure our systems in a completely automated > hands-off manner. We have 200+ FreeBSD systems and we definitely need an > automated process. We already have it working fine, but without > mirroring. We can upgrade doezens of systems at a time simply by making > them boot from our PXE server. We now need to tweak this process so that > we can establish the mirrored file systems as part of the automated > install. What I've done in the past is skip sysinstall alltogether and just boot of an NFS root. Then use custom scripts for the slicing/partitioning/ mirroring, copy a minimal system to disk and pkg_add the rest. Would be nice to do all this with install.cfg though. Please let me know when you get this working. Ruben From bsd.quest at googlemail.com Tue Nov 18 05:30:13 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Tue Nov 18 05:30:20 2008 Subject: =?utf-8?b?0J7RgtCy0LXRgjo=?= KLD loading, liking In-Reply-To: <20081117204407.GY81783@hoeg.nl> References: <671bb5fc0811151927j6dcb5155oeffbf4cc95cbccb6@mail.gmail.com> <20081116100923.GU81783@hoeg.nl> <20081117113522.GA2732@debian.samsung.router> <65f70ae30811170735i60891633ja0791f3d73cee188@mail.gmail.com> <20081117204407.GY81783@hoeg.nl> Message-ID: <20081118132944.GA2967@debian.samsung.router> On Mon, Nov 17, 2008 at 09:43:44PM +0100, Ed Schouten wrote: > * Aleksandr Litvinov wrote: > > Hello, > > You can receive a little information about KLD from the book > > "designing BSD rootkits". > > I don't own this book myself, but a colleague at Snow B.V. once showed > it to me. I only looked through it a couple of minutes, but it seemed > like a book nice to have. It also shows some techniques on how to hide > KLD's. I have this book. It shows some techniques, but it doesn't explain many things. And for KLD loading it gives only easy examples without explaining how KLD-Loader works. It's not absolutely necessary to bye this book. There are some papers, which explain the topics of the book very well: 1. Fun and Games with FreeBSD Kernel Modules http://www.r4k.net/mod/fbsdfun.html 2. Attacking FreeBSD with Kernel Modules: http://packetstormsecurity.org/papers/unix/bsdkern.htm > > -- > Ed Schouten > WWW: http://80386.nl/ -- Alexej Sokolov From psteele at maxiscale.com Tue Nov 18 07:55:33 2008 From: psteele at maxiscale.com (Peter Steele) Date: Tue Nov 18 07:55:40 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <200811182111.02367.doconnor@gsoft.com.au> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> <200811182111.02367.doconnor@gsoft.com.au> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240C920@polaris.maxiscale.com> >I believe you modify /usr/src/release/${ARCH}/boot_crunch.conf to do this. > >I haven't actually tried though... > >I think it would be possible to have a 'GEOM' menu that you can run prior to fdisk, label, etc that would allow you to >do some basic stuff like this. > >While the sysinstall code is a bit fugly it's not that difficult to hack on (speaking from limited experience :) Hmmm. I hadn't planned on actually creating a custom sysinstall but I guess that's another way we could approach this. I have some research to do... From psteele at maxiscale.com Tue Nov 18 08:00:25 2008 From: psteele at maxiscale.com (Peter Steele) Date: Tue Nov 18 08:00:32 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <20081118085739.GA95079@ei.bzerk.org> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B505B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F23B5102@polaris.maxiscale.com> <20081118085739.GA95079@ei.bzerk.org> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240C922@polaris.maxiscale.com> >What I've done in the past is skip sysinstall alltogether and just boot >of an NFS root. Then use custom scripts for the slicing/partitioning/ >mirroring, copy a minimal system to disk and pkg_add the rest. >Would be nice to do all this with install.cfg though. Please let me know >when you get this working. I thought of doing something like this as well. I'll have to investigate this as another option to this problem. Thanks for the feedback guys. From ed at 80386.nl Tue Nov 18 13:34:11 2008 From: ed at 80386.nl (Ed Schouten) Date: Tue Nov 18 13:34:18 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081028081154.GQ6808@hoeg.nl> References: <20081028081154.GQ6808@hoeg.nl> Message-ID: <20081118213410.GA81783@hoeg.nl> Skipped content of type multipart/mixed-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081118/7cc3077d/attachment.pgp From psteele at maxiscale.com Tue Nov 18 13:39:37 2008 From: psteele at maxiscale.com (Peter Steele) Date: Tue Nov 18 13:39:44 2008 Subject: What are proper install.cfg for configuring multiple slices? Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240C97F@polaris.maxiscale.com> I want to do an automated sysinstall through an install.cfg script and the script partition the install disk into three slices. I've been going through various tests trying to figure out what the proper directives are but I haven't had much luck, and I can't find any good examples. Here is a snippet of my config file: disk=ad0 bootManager=standard partition=12582912 diskPartitionEditor partition=2097152 diskPartitionEditor partition=free diskPartitionEditor ad0s1-1=ufs 4194304 / ad0s1-2=ufs 4194304 /tmp ad0s1-3=ufs 4194304 /var ad0s2-1=swap 2097152 none ad0s3-1=ufs 4194304 none ad0s3-2=ufs 4194304 none ad0s3-3=ufs 0 none diskLabelEditor diskLabelCommit My intent here is to create three slices-one 6GB in size, another 1GB in size, and the third sized to consume the remaining free space. When I run this through sysinstall, it complains that it can't find the space for the partitions. It even complains that it can't find any free space. Because the slices don't get created, the subsequent label assignments fail as well. What is the proper commands for creating multiple slices in install.cfg? Another thing I'm having trouble with is partitioning more than one disk. I have four disks that I'd like to partition as part of the install.cfg script. In fact, I want to partition the four disks more or less identically (although only one should have an active root partition). Again though, if I try partitioning another disk after ad0, sysinstall complains about various things and the disk does not get partitioned. Can multiple disks be partitioned in this manner or does the step have to be done as a post-install operation? From doconnor at gsoft.com.au Tue Nov 18 14:13:12 2008 From: doconnor at gsoft.com.au (Daniel O'Connor) Date: Tue Nov 18 14:13:18 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240C920@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> <200811182111.02367.doconnor@gsoft.com.au> <2ACA3DE8F9758A48B8BE2C7A847F91F240C920@polaris.maxiscale.com> Message-ID: <200811190843.01321.doconnor@gsoft.com.au> On Wednesday 19 November 2008 02:25:12 Peter Steele wrote: > >I believe you modify /usr/src/release/${ARCH}/boot_crunch.conf to do > > this. > > >I haven't actually tried though... > > > >I think it would be possible to have a 'GEOM' menu that you can run > > prior to fdisk, label, etc that would allow you to > > >do some basic stuff like this. > > > >While the sysinstall code is a bit fugly it's not that difficult to > > hack on (speaking from limited experience :) > > Hmmm. I hadn't planned on actually creating a custom sysinstall but I > guess that's another way we could approach this. I have some research to > do... You wouldn't have to do so - you could just run a shell script from sysinstall and do what you want. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081118/5f017c28/attachment.pgp From david at catwhisker.org Tue Nov 18 14:26:29 2008 From: david at catwhisker.org (David Wolfskill) Date: Tue Nov 18 14:26:36 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081118213410.GA81783@hoeg.nl> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> Message-ID: <20081118214919.GM83287@bunrab.catwhisker.org> On Tue, Nov 18, 2008 at 10:34:10PM +0100, Ed Schouten wrote: > ... > One solution would be to let xconsole just display /var/log/messages. Errr... it may be rather a pathological case, but you might want to check the content of /etc/syslog.conf on the local machine before getting too carried away with that approach. For example, on my "firewall" box at home (where I really do not want to log anything to local disk files, though I do have a serial console on it): janus(6.4-P)[1] grep -v '^#' /etc/syslog.conf *.* @bunrab.catwhisker.org janus(6.4-P)[2] And then consider the fate of bunrab -- with stuff getting logged to /var/log/messages from various machines.... > ... > I'll discuss this with others to decide if we should take such an > approach. I'm not trying to be obstructionist, here. If the above case is really "too pathological to consider" -- or if it's a case of me bringing that fate upon myself, I suppose -- that's actually something I can live with. It would be nice to be forwarned about it, though. :-} Peace, david -- David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081118/1a801717/attachment.pgp From psteele at maxiscale.com Tue Nov 18 14:36:40 2008 From: psteele at maxiscale.com (Peter Steele) Date: Tue Nov 18 14:36:46 2008 Subject: How can I add new binaries to the mfsroot image? In-Reply-To: <200811190843.01321.doconnor@gsoft.com.au> References: <2ACA3DE8F9758A48B8BE2C7A847F91F23B502B@polaris.maxiscale.com> <200811182111.02367.doconnor@gsoft.com.au> <2ACA3DE8F9758A48B8BE2C7A847F91F240C920@polaris.maxiscale.com> <200811190843.01321.doconnor@gsoft.com.au> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240C996@polaris.maxiscale.com> >You wouldn't have to do so - you could just run a shell script from sysinstall and do what you want. That brings me back to my original problem. Yes, I can run a shell script from sysinstall, but gmirror isn't available in mfsroot, and adding gmirror to mfsroot isn't straightforward because it needs shared libraries. I think the best approach to use may very well to have a custom boot that mounts root from an NFS disk. Then I can run whatever commands I need without having to actually add anything to mfsroot... From nick at van-laarhoven.org Tue Nov 18 23:55:15 2008 From: nick at van-laarhoven.org (Nick Hibma) Date: Tue Nov 18 23:55:23 2008 Subject: Unicode USB strings conversion Message-ID: <200811190842.59377.nick@van-laarhoven.org> In the USB code (and I bet it is the same in the USB4BSD code) unicode characters in strings are converted in a very crude way to ASCII. As I have a user on the line who sees rubbish in his logs and when using usbctl/usbdevs/etc., I bet this is the problem. I'd like to try and fix this problem by using libkern/libiconv. 1) Is this the right approach to convert UTF8 to printable string in the kernel? 2) Is this needed at all in the short term future? I remember seeing attempts at making the kernel use UTF8. 3) Does anyone know of a good example in the code without me having to hunt through the kernel to find it? For reference: The code that needs replacing is: usbd_get_string(): s = buf; n = size / 2 - 1; for (i = 0; i < n && i < len - 1; i++) { c = UGETW(us.bString[i]); /* Convert from Unicode, handle buggy strings. */ if ((c & 0xff00) == 0) *s++ = c; else if ((c & 0x00ff) == 0 && swap) *s++ = c >> 8; else *s++ = '?'; } *s++ = 0; I haven't got the USB specs handy, but I believe that this is a simple way of converting LE and BE UTF8 to ASCII. Nick From hselasky at c2i.net Wed Nov 19 00:19:09 2008 From: hselasky at c2i.net (Hans Petter Selasky) Date: Wed Nov 19 00:19:17 2008 Subject: Unicode USB strings conversion In-Reply-To: <200811190842.59377.nick@van-laarhoven.org> References: <200811190842.59377.nick@van-laarhoven.org> Message-ID: <200811190921.13859.hselasky@c2i.net> On Wednesday 19 November 2008, Nick Hibma wrote: > In the USB code (and I bet it is the same in the USB4BSD code) unicode > characters in strings are converted in a very crude way to ASCII. As I have > a user on the line who sees rubbish in his logs and when using > usbctl/usbdevs/etc., I bet this is the problem. > > I'd like to try and fix this problem by using libkern/libiconv. > > 1) Is this the right approach to convert UTF8 to printable string in the > kernel? > > 2) Is this needed at all in the short term future? I remember seeing > attempts at making the kernel use UTF8. > > 3) Does anyone know of a good example in the code without me having to hunt > through the kernel to find it? > > For reference: The code that needs replacing is: > > usbd_get_string(): > > s = buf; > n = size / 2 - 1; > for (i = 0; i < n && i < len - 1; i++) { > c = UGETW(us.bString[i]); > /* Convert from Unicode, handle buggy strings. */ > if ((c & 0xff00) == 0) > *s++ = c; > else if ((c & 0x00ff) == 0 && swap) > *s++ = c >> 8; > else > *s++ = '?'; > } > *s++ = 0; > > I haven't got the USB specs handy, but I believe that this is a simple way > of converting LE and BE UTF8 to ASCII. Or you could try to search for a better language ID. Currently the USB stack (old and new) selects the first language ID in the language string. Probably there is an english language ID, but not as the first selection. --HPS From nick at van-laarhoven.org Wed Nov 19 00:42:36 2008 From: nick at van-laarhoven.org (Nick Hibma) Date: Wed Nov 19 00:42:44 2008 Subject: Unicode USB strings conversion In-Reply-To: <200811190921.13859.hselasky@c2i.net> References: <200811190842.59377.nick@van-laarhoven.org> <200811190921.13859.hselasky@c2i.net> Message-ID: <200811190930.25770.nick@van-laarhoven.org> > Or you could try to search for a better language ID. Currently the USB > stack (old and new) selects the first language ID in the language string. > Probably there is an english language ID, but not as the first selection. The first part of the string is correct, so I assume that they added some strange characters (like copyright) at the end. Besides, even in the English character set there might be strange characters, for example scandinavian characters in names. Nick From hselasky at c2i.net Wed Nov 19 01:19:11 2008 From: hselasky at c2i.net (Hans Petter Selasky) Date: Wed Nov 19 01:19:19 2008 Subject: Unicode USB strings conversion In-Reply-To: <200811190842.59377.nick@van-laarhoven.org> References: <200811190842.59377.nick@van-laarhoven.org> Message-ID: <200811190921.13859.hselasky@c2i.net> On Wednesday 19 November 2008, Nick Hibma wrote: > In the USB code (and I bet it is the same in the USB4BSD code) unicode > characters in strings are converted in a very crude way to ASCII. As I have > a user on the line who sees rubbish in his logs and when using > usbctl/usbdevs/etc., I bet this is the problem. > > I'd like to try and fix this problem by using libkern/libiconv. > > 1) Is this the right approach to convert UTF8 to printable string in the > kernel? > > 2) Is this needed at all in the short term future? I remember seeing > attempts at making the kernel use UTF8. > > 3) Does anyone know of a good example in the code without me having to hunt > through the kernel to find it? > > For reference: The code that needs replacing is: > > usbd_get_string(): > > s = buf; > n = size / 2 - 1; > for (i = 0; i < n && i < len - 1; i++) { > c = UGETW(us.bString[i]); > /* Convert from Unicode, handle buggy strings. */ > if ((c & 0xff00) == 0) > *s++ = c; > else if ((c & 0x00ff) == 0 && swap) > *s++ = c >> 8; > else > *s++ = '?'; > } > *s++ = 0; > > I haven't got the USB specs handy, but I believe that this is a simple way > of converting LE and BE UTF8 to ASCII. Or you could try to search for a better language ID. Currently the USB stack (old and new) selects the first language ID in the language string. Probably there is an english language ID, but not as the first selection. --HPS From yanefbsd at gmail.com Wed Nov 19 02:34:01 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Wed Nov 19 02:34:08 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081118214919.GM83287@bunrab.catwhisker.org> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> Message-ID: <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> On Tue, Nov 18, 2008 at 1:49 PM, David Wolfskill wrote: > On Tue, Nov 18, 2008 at 10:34:10PM +0100, Ed Schouten wrote: >> ... >> One solution would be to let xconsole just display /var/log/messages. > > Errr... it may be rather a pathological case, but you might want to > check the content of /etc/syslog.conf on the local machine before > getting too carried away with that approach. > > For example, on my "firewall" box at home (where I really do not want to > log anything to local disk files, though I do have a serial console on it): > > janus(6.4-P)[1] grep -v '^#' /etc/syslog.conf > *.* @bunrab.catwhisker.org > janus(6.4-P)[2] > > And then consider the fate of bunrab -- with stuff getting logged to > /var/log/messages from various machines.... > >> ... >> I'll discuss this with others to decide if we should take such an >> approach. > > I'm not trying to be obstructionist, here. If the above case is really > "too pathological to consider" -- or if it's a case of me bringing that > fate upon myself, I suppose -- that's actually something I can live > with. It would be nice to be forwarned about it, though. :-} > > Peace, > david Uh, I second that. /var/log/messages shouldn't necessarily be accessible by non-root users. Also, OSX 10.5 protects against non-root access to dmesg. Not saying we should go that far, but it's already being implemented, so I don't see any harm in hiding the contents of `messages', as required by the sysadmin. -Garrett From koitsu at FreeBSD.org Wed Nov 19 02:47:33 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Wed Nov 19 02:47:41 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> Message-ID: <20081119104731.GA83366@icarus.home.lan> On Wed, Nov 19, 2008 at 02:02:42AM -0800, Garrett Cooper wrote: > On Tue, Nov 18, 2008 at 1:49 PM, David Wolfskill wrote: > > On Tue, Nov 18, 2008 at 10:34:10PM +0100, Ed Schouten wrote: > >> ... > >> One solution would be to let xconsole just display /var/log/messages. > > > > Errr... it may be rather a pathological case, but you might want to > > check the content of /etc/syslog.conf on the local machine before > > getting too carried away with that approach. > > > > For example, on my "firewall" box at home (where I really do not want to > > log anything to local disk files, though I do have a serial console on it): > > > > janus(6.4-P)[1] grep -v '^#' /etc/syslog.conf > > *.* @bunrab.catwhisker.org > > janus(6.4-P)[2] > > > > And then consider the fate of bunrab -- with stuff getting logged to > > /var/log/messages from various machines.... > > > >> ... > >> I'll discuss this with others to decide if we should take such an > >> approach. > > > > I'm not trying to be obstructionist, here. If the above case is really > > "too pathological to consider" -- or if it's a case of me bringing that > > fate upon myself, I suppose -- that's actually something I can live > > with. It would be nice to be forwarned about it, though. :-} > > > > Peace, > > david > > Uh, I second that. /var/log/messages shouldn't necessarily be > accessible by non-root users. Also, OSX 10.5 protects against non-root > access to dmesg. Not saying we should go that far, but it's already > being implemented, so I don't see any harm in hiding the contents of > `messages', as required by the sysadmin. Footnote (not really applicable to the thread, but I want to point it out to users/admins reading): inhibiting users viewing the kernel message buffer (dmesg) can be accomplished by setting the security.bsd.unprivileged_read_msgbuf sysctl to 0. However, note that this can piss users off. We have numerous users on our system who rely on this information to see if anything "weird" is going on with the box. I set that sysctl one day (see below for why), and I got flames in my mailbox within 48 hours. Just something to keep in mind if you have technically-savvy users. There's a known "issue" with the kernel message buffer though: it's not NULL'd out upon reboot. Meaning, in some cases (depends on the BIOS or system), the kernel message buffer from single-user mode is retained even after a reboot! A user can then do "dmesg" and see all the nifty stuff you've done during single-user, which could include unencrypted passwords if mergemaster was tinkering with passwd/master.passwd, etc.. I've brought this up before, and people said "Yeah, we know, moving on". Rink Springer created a patch where the kernel message buffer will start with NULL to keep this from happening, but it needs to be made into a loader.conf tunable. Also, /var/log/messages is explicitly set to 0644 in newsyslog.conf. If people want to debate that, be my guest. I'm not sure what "security hole" we'd be plugging if it was set to 0600, especially given that many userland programs use the LOG_NOTICE facility in syslog. If people want to debate those default perms, be my guest. I would rather people debate the default syslog.conf layout altogether; I'm surprised we haven't moved to syslog-ng (as part of the base system) by now. :-) -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From unixmania at gmail.com Wed Nov 19 03:33:07 2008 From: unixmania at gmail.com (Carlos A. M. dos Santos) Date: Wed Nov 19 03:33:14 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081118213410.GA81783@hoeg.nl> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> Message-ID: On Tue, Nov 18, 2008 at 7:34 PM, Ed Schouten wrote: > Hello Carlos, others, > > * Ed Schouten wrote: >> About the /dev/console issues: Robert Watson and I discussed this some >> time ago on IRC and what I did in HEAD (not RELENG_7) was that I changed >> TIOCCONS not to take a look at the permissions of /dev/console, but we >> changed it to use priv_check(). This means that right now you can only >> call TIOCCONS as root. I can't really understand why the problem exists >> on RELENG_7. >> >> About making xconsole setuid: I've read the messages you mentioned, but >> I think we could just alter console to call TIOCCONS and just drop >> privileges. An even better solution would be to just get rid of TIOCCONS >> and invent a better solution to capture syslog messages. I can't really >> understand why we want to abuse TTY's to do this. >> >> So I can't say we're working on this, but at least I can confirm the >> issue. > > One solution would be to let xconsole just display /var/log/messages. > There shouldn't be a valid reason to let syslogd print messages to > /dev/console and capture them again using TIOCCONS. We could just > instruct xconsole to read its data from the log files. > > If you save the attached patch as /usr/ports/x11/xconsole/files/ > patch-xconsole.c (create directory first) and recompile xconsole, it > will use the log file. > > I'll discuss this with others to decide if we should take such an > approach. It is not necessary to patch xconsole to accomplish this. Using the -file command line argument would be enough. Be warned, however, that 1. messages sent straight to /dev/console will not show up at the xconsole window; 2. with large files it will become slow and consume lots of memory, because it will load the entire contents of /mar/log/messages to its text buffer; 3. it will show *all* messages, not only the urgent ones, which is not necessarily the desired behavior; 4. it will stop working upon log rotation. -- cd /usr/ports/sysutils/life make clean From vadim_nuclight at mail.ru Wed Nov 19 03:37:18 2008 From: vadim_nuclight at mail.ru (Vadim Goncharov) Date: Wed Nov 19 03:37:26 2008 Subject: cosum: Checkout verification PoC References: <200809222233.26053.max@love2party.net> Message-ID: Hi Max Laier! On Mon, 22 Sep 2008 22:33:25 +0200; Max Laier wrote about 'cosum: Checkout verification PoC': > the attached script will generate md5 and sha256 checksums of a checkout and > try to find the corresponding svn-revision. This can help to verify that your > checkout from cvsupX.yy.freebsd.org is authentic. Not that there is reason to > believe that we have compromised cvsup-servers. This is just something I've > been toying with and wanted to let you know to see if people find the idea > interesting. I'd also be interested in reviews of the concept (note that I > know that https would be a good idea, I just cba to setup a certificate). > The coverage currently is head and stable/{6,7} svn revision 179451:183186 > (i.e. since the first svn commit up to "2008-09-19 16:51:41 +0200". I don't > yet have a cronjob in place to generate new checksums, so this will become > less useful quick. If people do find it interesting, however, I could > certainly roll something. > As you can see, the script is ready to checksum cvs and svn checkouts. If you > obtain your checkout from some local git/hg/svk/... mirror you must modify the > find excludes accordingly. > Let me know what you think. This is a good solution for our users caring about security. I think such definitely should be incorporated into base system and server-side support be provided at freebsd.org on official basis. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From takawata at init-main.com Wed Nov 19 03:42:03 2008 From: takawata at init-main.com (Takanori Watanabe) Date: Wed Nov 19 03:42:11 2008 Subject: Core i7 anyone else? Message-ID: <200811191144.mAJBi3Lg004559@sana.init-main.com> Hi, I recently bought Core i7 machine(for 145,000JPY: about $1500) and sometimes hangs up oddly. When in the state, some specific process only works and replys ping, but not reply any useful information. I suspect it may caused by CPU power management, so I cut almost all CPU power management feature on BIOS parameter. Are there any people encouterd such trouble? And on this machine build world in SCHED_ULE(15min.) is slower than SCHED_4BSD(12min.). ===dmesg=== http://www.init-main.com/corei7.dmesg or http://pastebin.com/m187f77aa (if host is down) =====DSDT==== http://www.init-main.com/corei7.asl or http://pastebin.com/m6879984a ==some sysctls== hw.machine: i386 hw.model: Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz hw.ncpu: 8 hw.byteorder: 1234 hw.physmem: 3202322432 hw.usermem: 2956083200 hw.pagesize: 4096 hw.floatingpoint: 1 hw.machine_arch: i386 hw.realmem: 3211264000 == machdep.enable_panic_key: 0 machdep.adjkerntz: -32400 machdep.wall_cmos_clock: 1 machdep.disable_rtc_set: 0 machdep.disable_mtrrs: 0 machdep.guessed_bootdev: 2686451712 machdep.idle: acpi machdep.idle_available: spin, mwait, mwait_hlt, hlt, acpi, machdep.hlt_cpus: 0 machdep.prot_fault_translation: 0 machdep.panic_on_nmi: 1 machdep.kdb_on_nmi: 1 machdep.tsc_freq: 2684011396 machdep.i8254_freq: 1193182 machdep.acpi_timer_freq: 3579545 machdep.acpi_root: 1024240 machdep.hlt_logical_cpus: 0 machdep.logical_cpus_mask: 254 machdep.hyperthreading_allowed: 1 == kern.sched.preemption: 0 kern.sched.topology_spec: 0, 1, 2, 3, 4, 5, 6, 7 kern.sched.steal_thresh: 3 kern.sched.steal_idle: 1 kern.sched.steal_htt: 1 kern.sched.balance_interval: 133 kern.sched.balance: 1 kern.sched.affinity: 1 kern.sched.idlespinthresh: 4 kern.sched.idlespins: 10000 kern.sched.static_boost: 160 kern.sched.preempt_thresh: 0 kern.sched.interact: 30 kern.sched.slice: 13 kern.sched.name: ULE === From koitsu at FreeBSD.org Wed Nov 19 03:47:17 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Wed Nov 19 03:47:29 2008 Subject: Core i7 anyone else? In-Reply-To: <200811191144.mAJBi3Lg004559@sana.init-main.com> References: <200811191144.mAJBi3Lg004559@sana.init-main.com> Message-ID: <20081119114714.GA85533@icarus.home.lan> On Wed, Nov 19, 2008 at 08:44:03PM +0900, Takanori Watanabe wrote: > Hi, I recently bought Core i7 machine(for 145,000JPY: about $1500) > and sometimes hangs up oddly. > When in the state, some specific process only works and > replys ping, but not reply any useful information. > > I suspect it may caused by CPU power management, so I cut > almost all CPU power management feature on BIOS parameter. > > Are there any people encouterd such trouble? > And on this machine build world in SCHED_ULE(15min.) is slower > than SCHED_4BSD(12min.). > > > ===dmesg=== > http://www.init-main.com/corei7.dmesg > or > http://pastebin.com/m187f77aa > (if host is down) > > =====DSDT==== > http://www.init-main.com/corei7.asl > or > http://pastebin.com/m6879984a > > ==some sysctls== > hw.machine: i386 > hw.model: Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz > hw.ncpu: 8 > hw.byteorder: 1234 > hw.physmem: 3202322432 > hw.usermem: 2956083200 > hw.pagesize: 4096 > hw.floatingpoint: 1 > hw.machine_arch: i386 > hw.realmem: 3211264000 > == > machdep.enable_panic_key: 0 > machdep.adjkerntz: -32400 > machdep.wall_cmos_clock: 1 > machdep.disable_rtc_set: 0 > machdep.disable_mtrrs: 0 > machdep.guessed_bootdev: 2686451712 > machdep.idle: acpi > machdep.idle_available: spin, mwait, mwait_hlt, hlt, acpi, > machdep.hlt_cpus: 0 > machdep.prot_fault_translation: 0 > machdep.panic_on_nmi: 1 > machdep.kdb_on_nmi: 1 > machdep.tsc_freq: 2684011396 > machdep.i8254_freq: 1193182 > machdep.acpi_timer_freq: 3579545 > machdep.acpi_root: 1024240 > machdep.hlt_logical_cpus: 0 > machdep.logical_cpus_mask: 254 > machdep.hyperthreading_allowed: 1 > == > kern.sched.preemption: 0 > kern.sched.topology_spec: > > 0, 1, 2, 3, 4, 5, 6, 7 > > > > > kern.sched.steal_thresh: 3 > kern.sched.steal_idle: 1 > kern.sched.steal_htt: 1 > kern.sched.balance_interval: 133 > kern.sched.balance: 1 > kern.sched.affinity: 1 > kern.sched.idlespinthresh: 4 > kern.sched.idlespins: 10000 > kern.sched.static_boost: 160 > kern.sched.preempt_thresh: 0 > kern.sched.interact: 30 > kern.sched.slice: 13 > kern.sched.name: ULE > === When building world/kernel, do you see odd behaviour (on CURRENT) such as the load average being absurdly high, or processes (anything; sh, make, mutt, etc.) getting stuck in bizarre states? These things are what caused my buildworld/buildkernel times to increase (compared to RELENG_7). I was using ULE entirely (on CURRENT and RELENG_7), but did not try 4BSD. I documented my experience. http://wiki.freebsd.org/JeremyChadwick/Bizarre_CURRENT_experience I have no idea if your problem is the same as mine. This is purely speculative on my part. (And readers of that Wiki article should note that the problem was not hardware-related) -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From ivoras at freebsd.org Wed Nov 19 03:58:16 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Wed Nov 19 03:58:23 2008 Subject: Core i7 anyone else? In-Reply-To: <200811191144.mAJBi3Lg004559@sana.init-main.com> References: <200811191144.mAJBi3Lg004559@sana.init-main.com> Message-ID: <4923FF7E.1080101@freebsd.org> Takanori Watanabe wrote: > Hi, I recently bought Core i7 machine(for 145,000JPY: about $1500) > and sometimes hangs up oddly. > When in the state, some specific process only works and > replys ping, but not reply any useful information. > > I suspect it may caused by CPU power management, so I cut > almost all CPU power management feature on BIOS parameter. > > Are there any people encouterd such trouble? > And on this machine build world in SCHED_ULE(15min.) is slower > than SCHED_4BSD(12min.). I don't know but this: > ===dmesg=== > http://www.init-main.com/corei7.dmesg > or > http://pastebin.com/m187f77aa > (if host is down) CPU: Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz (2684.00-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106a4 Stepping = 4 Features=0xbfebfbff Features2=0x98e3bd AMD Features=0x28100000 AMD Features2=0x1 Cores per package: 8 Logical CPUs per core: 2 real memory = 3211264000 (3062 MB) avail memory = 3143983104 (2998 MB) ACPI APIC Table: <7522MS A7522100> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 is a bit in conflict with this: > kern.sched.topology_spec: > > 0, 1, 2, 3, 4, 5, 6, 7 > > > From what I know of its architecture i7 has hyperthreading - i.e. the CPU has 4 "real" cores which are hyperthreaded, so you get 8 cores total. It probably also includes a different way of enumerating its topology which might have caused wrong topology detection and your slowdown in buildworld. (the CPU also has L3 cache, but I think it's not looked up in topology detection). I don't know it this particular error could be responsible for your lockups - probably not. The CPU also introduces some big changes in power management (dynamic powerdown of individual cores) which could cause them - but I can't help you there. Are you sure it's not something trivial like overheating? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081119/c476043b/signature.pgp From marc.loerner at hob.de Wed Nov 19 06:27:04 2008 From: marc.loerner at hob.de (Marc =?iso-8859-1?q?L=F6rner?=) Date: Wed Nov 19 06:27:11 2008 Subject: ide with DMA and ram > 4GB In-Reply-To: <200811131502.04370.jhb@freebsd.org> References: <200811121823.14400.marc.loerner@hob.de> <200811131502.04370.jhb@freebsd.org> Message-ID: <200811191507.22455.marc.loerner@hob.de> On Thursday 13 November 2008 21:02, John Baldwin wrote: > On Wednesday 12 November 2008 12:23:14 pm Marc L?rner wrote: > > Hello, > > I just stepped over a problem with my IDE disk running in DMA-mode > > and having more than 4GB of RAM. > > It seems that the whole way down GEOM, ata-disk, ata-dma never is checked > > whether physical address of buffer is less than 4GB an so fits in 32bits. > > => when PRD is set the address is rigorously truncated to fit into 32bit, > > with buffer < 4GB all is quite fine. > > bus_dmamap_load() in ata-dma.c should result in bounce pages being > allocated and having the data copied to pages below 4GB and having those > addresses passed to the callback and stored in the PRD. Thanks for pointing this out! So it seems that bounce-pages are counted with helper-function run_filter. But this function does never return 1 with dma-pages not lying in range (e.g. paddr > 4GB) but being aligned. Did nobody else have problems on 64bit ide-dma, or is it already working, but I didn't grasp functionality, yet? For now, I came around this by adding a flag in "/sys/bus_dma.h" and setting this flag on tag-creation. So I now can check in run_filter whether I'm doing an ide-dma, relying on this and on check whether I got paddr > 4GB, I then can tell bus_dmamap_load to use bounce-pages. Any thoughts or comments? Regards, Marc Loerner From psteele at maxiscale.com Wed Nov 19 09:14:40 2008 From: psteele at maxiscale.com (Peter Steele) Date: Wed Nov 19 09:14:47 2008 Subject: What are proper install.cfg for configuring multiple slices? In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240C97F@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240C97F@polaris.maxiscale.com> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CA25@polaris.maxiscale.com> >I want to do an automated sysinstall through an install.cfg script and >the script partition the install disk into three slices. I've been going >through various tests trying to figure out what the proper directives >are but I haven't had much luck, and I can't find any good examples. After a lot of experimenting, my impression is that sysinstall simply doesn't support multiple slice installations. It works to a point, but I get some unexpected errors, e.g. Unable to make device node for /dev/ad0s1a in /dev and after the partitioning is complete, there are funky entries under /dev: /dev/ad0c /dev/ad0cs1 /dev/ad0cs2 /dev/ad0cs3 /dev/ad0s1 /dev/ad0s2 /dev/ad0s3 There should be entries such as /dev/ad0s1a and so on, but these do not get created. I've been unable to find even one example of how to formulate multiple partitions in install.cfg, but I'm pretty sure I'm doing it right, based on the sysinstall docs. Does anyone have any experience with this? From jhs at berklix.org Wed Nov 19 10:00:18 2008 From: jhs at berklix.org (Julian Stacey) Date: Wed Nov 19 10:00:27 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? Message-ID: <200811191724.mAJHOL8r062364@fire.js.berklix.net> Hi hackers, Maybe Some of you might suggest some software I might install, Wiki I guess. ? I got zero response from ports@, I could use some reccomendations please. PS From http://wiki.freebsd.org/HelpContents I tried cd /usr/ports/www ; vi *iki*/pkg-descr or is /usr/ports/www/moinmoin the way to go ? Thanks. ----------- Subject: Reccomendation for ports for web based club events forthcoming diary ? Can anyone reccomend some ports to install on a FreeBSD web server, for a club of mostly non technical people, to support: - All club members can add events to a forthcoming calendar, - All club members can request server to prepare a listing of next next upcoming events, to download (probably in PDF, or perhaps tbl to a pipe or ? - A list of moderators can delete fake events from robots & the malicious. - Preferably moderators should not themselves be capable of deleting logged event submission, but only capable of deleting events formatted to the ouput printable programme sheet. (To autopsy for suspect rogue moderators) - I guess first entry criteria might be a fuzzy picture for human to decode password from). 2nd might be mail return for confirm password, - & 3rd, A majordomo (later mailman) maintained list of club members & moderators etc is available for automated validation. - I hope there will be some packages available, http & probably wiki based etc, that will come close enough ? I'm hoping this has been done often enough that people can suggest names of ports already existing ? If not I dont mind creating a port if I have to, but dont want to write something from scratch. PS - - I've had apache up for years, but no wiki yet, so if any tips, shout please, even if just RTFM URL=.... :-) - - Web based forums I don't care about, but others may, so I suppose if some software does & does not support web forums, it'd be good to know. Suggestions welcome please ! Thanks in advance. Cheers, Julian - -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From yanefbsd at gmail.com Wed Nov 19 10:39:33 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Wed Nov 19 10:39:40 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? In-Reply-To: <200811191724.mAJHOL8r062364@fire.js.berklix.net> References: <200811191724.mAJHOL8r062364@fire.js.berklix.net> Message-ID: <7d6fde3d0811191008u75552bcbkbca2c5c302435d94@mail.gmail.com> On Wed, Nov 19, 2008 at 9:24 AM, Julian Stacey wrote: > Hi hackers, > Maybe Some of you might suggest some software I might install, Wiki I guess. ? > I got zero response from ports@, I could use some reccomendations please. > PS From http://wiki.freebsd.org/HelpContents I tried > cd /usr/ports/www ; vi *iki*/pkg-descr > or is /usr/ports/www/moinmoin the way to go ? > Thanks. > ----------- > > Subject: Reccomendation for ports for web based club events forthcoming diary ? > > Can anyone reccomend some ports to install on a FreeBSD web server, > for a club of mostly non technical people, to support: > - All club members can add events to a forthcoming calendar, > - All club members can request server to prepare a listing > of next next upcoming events, to download (probably in PDF, > or perhaps tbl to a pipe or ? > - A list of moderators can delete fake events from robots & the malicious. > - Preferably moderators should not themselves be capable of > deleting logged event submission, but only capable of deleting > events formatted to the ouput printable programme sheet. (To > autopsy for suspect rogue moderators) > - I guess first entry criteria might be a fuzzy picture for human > to decode password from). 2nd might be mail return for confirm password, > - & 3rd, A majordomo (later mailman) maintained list of club members & > moderators etc is available for automated validation. > - I hope there will be some packages available, > http & probably wiki based etc, that will come close enough ? > I'm hoping this has been done often enough that people can suggest > names of ports already existing ? If not I dont mind creating a > port if I have to, but dont want to write something from scratch. > > PS > - - I've had apache up for years, but no wiki yet, so if any tips, > shout please, even if just RTFM URL=.... :-) > - - Web based forums I don't care about, but others may, so I suppose if some > software does & does not support web forums, it'd be good to know. > > Suggestions welcome please ! Thanks in advance. > > Cheers, > Julian Julian, FWIW, ports@ or questions@ would be better lists than hackers@. Thanks, -Garrett From koitsu at FreeBSD.org Wed Nov 19 10:44:10 2008 From: koitsu at FreeBSD.org (Jeremy Chadwick) Date: Wed Nov 19 10:44:25 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? In-Reply-To: <7d6fde3d0811191008u75552bcbkbca2c5c302435d94@mail.gmail.com> References: <200811191724.mAJHOL8r062364@fire.js.berklix.net> <7d6fde3d0811191008u75552bcbkbca2c5c302435d94@mail.gmail.com> Message-ID: <20081119184405.GA4751@icarus.home.lan> On Wed, Nov 19, 2008 at 10:08:03AM -0800, Garrett Cooper wrote: > On Wed, Nov 19, 2008 at 9:24 AM, Julian Stacey wrote: > > Hi hackers, > > Maybe Some of you might suggest some software I might install, Wiki I guess. ? > > I got zero response from ports@, I could use some reccomendations please. > > PS From http://wiki.freebsd.org/HelpContents I tried > > cd /usr/ports/www ; vi *iki*/pkg-descr > > or is /usr/ports/www/moinmoin the way to go ? > > Thanks. > > ----------- > > > > Subject: Reccomendation for ports for web based club events forthcoming diary ? > > > > Can anyone reccomend some ports to install on a FreeBSD web server, > > for a club of mostly non technical people, to support: > > - All club members can add events to a forthcoming calendar, > > - All club members can request server to prepare a listing > > of next next upcoming events, to download (probably in PDF, > > or perhaps tbl to a pipe or ? > > - A list of moderators can delete fake events from robots & the malicious. > > - Preferably moderators should not themselves be capable of > > deleting logged event submission, but only capable of deleting > > events formatted to the ouput printable programme sheet. (To > > autopsy for suspect rogue moderators) > > - I guess first entry criteria might be a fuzzy picture for human > > to decode password from). 2nd might be mail return for confirm password, > > - & 3rd, A majordomo (later mailman) maintained list of club members & > > moderators etc is available for automated validation. > > - I hope there will be some packages available, > > http & probably wiki based etc, that will come close enough ? > > I'm hoping this has been done often enough that people can suggest > > names of ports already existing ? If not I dont mind creating a > > port if I have to, but dont want to write something from scratch. > > > > PS > > - - I've had apache up for years, but no wiki yet, so if any tips, > > shout please, even if just RTFM URL=.... :-) > > - - Web based forums I don't care about, but others may, so I suppose if some > > software does & does not support web forums, it'd be good to know. > > > > Suggestions welcome please ! Thanks in advance. > > > > Cheers, > > Julian > > Julian, > FWIW, ports@ or questions@ would be better lists than hackers@. He asked this on -ports only 2 days ago and received no response. So he appears to be going from list to list hoping someone will answer him. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From llc2w at virginia.edu Wed Nov 19 11:14:45 2008 From: llc2w at virginia.edu (L Campbell) Date: Wed Nov 19 11:15:43 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? In-Reply-To: <200811191724.mAJHOL8r062364@fire.js.berklix.net> References: <200811191724.mAJHOL8r062364@fire.js.berklix.net> Message-ID: <792298050811191049y1cc01679h9271c9decd2109b7@mail.gmail.com> On Wed, Nov 19, 2008 at 12:24 PM, Julian Stacey wrote: > > Hi hackers, > Maybe Some of you might suggest some software I might install, Wiki I guess. ? > I got zero response from ports@, I could use some reccomendations please. > > - - I've had apache up for years, but no wiki yet, so if any tips, > shout please, even if just RTFM URL=.... :-) http://en.wikipedia.org/wiki/List_of_wiki_software http://en.wikipedia.org/wiki/Comparison_of_wiki_software From jhs at berklix.org Wed Nov 19 14:18:26 2008 From: jhs at berklix.org (Julian Stacey) Date: Wed Nov 19 14:18:34 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? In-Reply-To: Your message "Wed, 19 Nov 2008 13:49:39 EST." <792298050811191049y1cc01679h9271c9decd2109b7@mail.gmail.com> Message-ID: <200811192218.mAJMIG1t066653@fire.js.berklix.net> > > - - I've had apache up for years, but no wiki yet, so if any tips, > > shout please, even if just RTFM URL=.... :-) > > http://en.wikipedia.org/wiki/List_of_wiki_software > http://en.wikipedia.org/wiki/Comparison_of_wiki_software Great ! Thanks a lot ! Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From ken at mthelicon.com Wed Nov 19 17:46:27 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Wed Nov 19 17:46:33 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT Message-ID: Hi everyone, I am having difficulties rebuilding the world after some patches were made today. I was wondering if anyone else is experiencing the same troubles? ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x0 -o gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/lib/crt0.o zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No such file or directory *** Error code 1 Stop in /usr/src/sys/boot/i386/gptzfsboot. *** Error code 1 Stop in /usr/src/sys/boot/i386. *** Error code 1 Stop in /usr/src/sys/boot. *** Error code 1 Stop in /usr/src/sys. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. feathers# From peterjeremy at optushome.com.au Wed Nov 19 22:39:41 2008 From: peterjeremy at optushome.com.au (Peter Jeremy) Date: Wed Nov 19 22:39:47 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081119104731.GA83366@icarus.home.lan> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> Message-ID: <20081120063936.GU51761@server.vk2pj.dyndns.org> On 2008-Nov-19 02:47:31 -0800, Jeremy Chadwick wrote: >There's a known "issue" with the kernel message buffer though: it's not >NULL'd out upon reboot. This is deliberate. If the system panics, stuff that was in the message buffer (and might not be on disk) can be read when the system reboots. If there is no crashdump, this might be the only record of what happened. > Meaning, in some cases (depends on the BIOS or >system), the kernel message buffer from single-user mode is retained >even after a reboot! A user can then do "dmesg" and see all the nifty >stuff you've done during single-user, which could include unencrypted >passwords if mergemaster was tinkering with passwd/master.passwd, etc.. There shouldn't be unencrypted passwords, though there might be encrypted passwords visible. >Rink Springer created a patch where the kernel message buffer will start >with NULL to keep this from happening, but it needs to be made into a >loader.conf tunable. I hope that never gets committed - it will make debugging kernel problems much harder. There is already a kern.msgbuf_clear sysctl and maybe people who are concerned about msgbuf leakage need to learn to use it. -- Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081120/2a9e173d/attachment.pgp From koitsu at freebsd.org Wed Nov 19 23:08:23 2008 From: koitsu at freebsd.org (Jeremy Chadwick) Date: Wed Nov 19 23:08:30 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081120063936.GU51761@server.vk2pj.dyndns.org> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> <20081120063936.GU51761@server.vk2pj.dyndns.org> Message-ID: <20081120070820.GA19307@icarus.home.lan> On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: > On 2008-Nov-19 02:47:31 -0800, Jeremy Chadwick wrote: > >There's a known "issue" with the kernel message buffer though: it's not > >NULL'd out upon reboot. > > This is deliberate. If the system panics, stuff that was in the > message buffer (and might not be on disk) can be read when the system > reboots. If there is no crashdump, this might be the only record of > what happened. That doesn't sound deliberate at all -- it sounds like a quirk that people (you?) are relying on. I do not think any piece of the FreeBSD system (e.g. savecore, etc.) relies on this behaviour. You're under the mentality that the information is *always* available after a panic/reboot -- it isn't. I have 4 different Supermicro motherboards (all from different years) which will "most of the time" lose the msgbuf after rebooting from single-user -- but sometimes the msgbuf is retained. And no, bad hardware is not responsible for the randomness of the problem. I think it's been discussed in the past how/why this can happen. It has to do with what each BIOS manufacturer chooses to do with some parts of memory during start-up. I'm sure the "Quick Boot" (e.g. no extensive memory test, which really doesn't test anything these days) option plays a role, and that option is enabled by default on all motherboards I've used in the past 10 years. > > Meaning, in some cases (depends on the BIOS or > >system), the kernel message buffer from single-user mode is retained > >even after a reboot! A user can then do "dmesg" and see all the nifty > >stuff you've done during single-user, which could include unencrypted > >passwords if mergemaster was tinkering with passwd/master.passwd, etc.. > > There shouldn't be unencrypted passwords, though there might be encrypted > passwords visible. Sorry, that's what I meant. The point is that a lot of things can go on in single-user mode which can/will disclose information or data in files which users do not have access to. Once the system is rebooted, a non-root user can do "dmesg -a" and see this buffer, getting access to data he/she normally does not have access to. Do you and I agree that this is in fact a security risk/problem? > >Rink Springer created a patch where the kernel message buffer will start > >with NULL to keep this from happening, but it needs to be made into a > >loader.conf tunable. > > I hope that never gets committed - it will make debugging kernel > problems much harder. There is already a kern.msgbuf_clear sysctl and > maybe people who are concerned about msgbuf leakage need to learn to > use it. And this sysctl is only usable *after* the kernel loads, which means you lose all of the messages shown from the time the kernel loads to the time the sysctl is set (e.g. hardware detected/configured). This is even less acceptable, IMHO. I would like to see Rink's patch committed, as long as the loader tunable defaults to *off* (e.g. current/historic behaviour). I'll also ask Rink to chime in here with his thoughts/opinions. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From kientzle at freebsd.org Wed Nov 19 23:13:05 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Wed Nov 19 23:13:13 2008 Subject: Unicode USB strings conversion In-Reply-To: <200811190842.59377.nick@van-laarhoven.org> References: <200811190842.59377.nick@van-laarhoven.org> Message-ID: <49250DFA.2050208@freebsd.org> Nick Hibma wrote: > In the USB code (and I bet it is the same in the USB4BSD code) unicode > characters in strings are converted in a very crude way to ASCII. As I have > a user on the line who sees rubbish in his logs and when using > usbctl/usbdevs/etc., I bet this is the problem. > > I'd like to try and fix this problem by using libkern/libiconv. > > 1) Is this the right approach to convert UTF8 to printable string in the > kernel? > > 2) Is this needed at all in the short term future? I remember seeing > attempts at making the kernel use UTF8. > > 3) Does anyone know of a good example in the code without me having to hunt > through the kernel to find it? > > For reference: The code that needs replacing is: > > usbd_get_string(): > > s = buf; > n = size / 2 - 1; > for (i = 0; i < n && i < len - 1; i++) { > c = UGETW(us.bString[i]); > /* Convert from Unicode, handle buggy strings. */ > if ((c & 0xff00) == 0) > *s++ = c; > else if ((c & 0x00ff) == 0 && swap) > *s++ = c >> 8; > else > *s++ = '?'; > } > *s++ = 0; > > I haven't got the USB specs handy, but I believe that this is a simple way > of converting LE and BE UTF8 to ASCII. First, get your terminology straight. It looks like UGETW() is returning 16-bit Unicode code points. That would be UTF-16, not UTF-8. UTF-8 is a popular multibyte encoding which uses 1 to 4 bytes per character. ASCII values (less than 128) get preserved, anything else gets encoded. There are two problems with UTF-16: First is determining the byte order. Second is that nobody displays UTF-16 directly. (Well, almost nobody.) The code above is fine if you're sure you're getting ASCII (it looks at each character and guesses the byte order) but is otherwise pretty lame. You didn't show the code that set the 'swap' variable. If you really want legible output, your best option by far is to really convert it to UTF8 and emit that. That still preserves ASCII, but gives a chance of viewing non-ASCII in a suitable terminal program. (And there are even a couple of folks looking into UTF8 support for syscons.) The basic UTF-16 to UTF-8 conversion is pretty simple: if (c < 0x7f) { *s++ = c; } else if (c < 0x7ff) { *s++ = 0xc0 | ((c >> 6) & 0x1f); *s++ = 0x80 | (c & 0x3f); } else if (c < 0xffff) { *s++ = 0xe0 | ((c >> 12) & 0x0f); *s++ = 0x80 | ((c >> 6) & 0x3f); *s++ = 0x80 | (c & 0x3f); } else { *s++ = 0xf0 | ((c >> 18) & 0x07); *s++ = 0x80 | ((c >> 12) & 0x3f); *s++ = 0x80 | ((c >> 6) & 0x3f); *s++ = 0x80 | (c & 0x3f); } This assumes that 'c' is a UTF-16 Unicode character in native byte order. If you really don't know the byte order, you'll need to find some way to guess. One way to guess is to assume that ASCII characters are common, in which case, you'll see things with the high order byte 0. In some environments, a "Byte-order mark" is used as the first character. This is character 0xFEFF. (The byte-swapped 0xFFFE is illegal, so if you see that, you know you've got the wrong byte order.) Good luck! Tim From neldredge at math.ucsd.edu Wed Nov 19 23:48:37 2008 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Wed Nov 19 23:48:43 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081120070820.GA19307@icarus.home.lan> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> <20081120063936.GU51761@server.vk2pj.dyndns.org> <20081120070820.GA19307@icarus.home.lan> Message-ID: On Wed, 19 Nov 2008, Jeremy Chadwick wrote: > On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: >> I hope that never gets committed - it will make debugging kernel >> problems much harder. There is already a kern.msgbuf_clear sysctl and >> maybe people who are concerned about msgbuf leakage need to learn to >> use it. > > And this sysctl is only usable *after* the kernel loads, which means > you lose all of the messages shown from the time the kernel loads to > the time the sysctl is set (e.g. hardware detected/configured). This is > even less acceptable, IMHO. But surely you can arrange that the contents are written out to /var/log/messages first? E.g. a sequence like - mount /var - write buffer contents via syslogd - clear buffer via sysctl - allow user logins This way the buffer is cleared before any unprivileged users get to do anything. No kernel changes needed, just a little tweaking of the init scripts at most. If you should have a crash and suspect there is useful data in the buffer, you can boot to single-user mode (avoiding the clear) and retrieve it manually. Seems like this should make everyone happy. -- Nate Eldredge neldredge@math.ucsd.edu From danny at cs.huji.ac.il Wed Nov 19 23:54:49 2008 From: danny at cs.huji.ac.il (Danny Braniss) Date: Wed Nov 19 23:54:56 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? In-Reply-To: <200811192218.mAJMIG1t066653@fire.js.berklix.net> References: <200811192218.mAJMIG1t066653@fire.js.berklix.net> Message-ID: > > > - - I've had apache up for years, but no wiki yet, so if any tips, > > > shout please, even if just RTFM URL=.... :-) > > > > http://en.wikipedia.org/wiki/List_of_wiki_software > > http://en.wikipedia.org/wiki/Comparison_of_wiki_software > > Great ! Thanks a lot ! Will you tell us which one you selected? danny > > Cheers, > Julian > -- > Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com > Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From des at des.no Thu Nov 20 01:53:09 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Thu Nov 20 01:53:16 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081120070820.GA19307@icarus.home.lan> (Jeremy Chadwick's message of "Wed, 19 Nov 2008 23:08:21 -0800") References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> <20081120063936.GU51761@server.vk2pj.dyndns.org> <20081120070820.GA19307@icarus.home.lan> Message-ID: <86r656oha4.fsf@ds4.des.no> Jeremy Chadwick writes: > Peter Jeremy writes: > > This is deliberate. If the system panics, stuff that was in the > > message buffer (and might not be on disk) can be read when the > > system reboots. If there is no crashdump, this might be the only > > record of what happened. > That doesn't sound deliberate at all -- it sounds like a quirk that > people (you?) are relying on. No, it is deliberate. Just because you don't like it doesn't mean we're morons. > You're under the mentality that the information is *always* available > after a panic/reboot -- it isn't. I have 4 different Supermicro > motherboards (all from different years) which will "most of the time" > lose the msgbuf after rebooting from single-user -- but sometimes the > msgbuf is retained. And no, bad hardware is not responsible for the > randomness of the problem. We know there are systems where it doesn't work. That doesn't mean it's not useful when it *does* work. There are also systems where it works reliably, including most SoC and SFF systems. > I think it's been discussed in the past how/why this can happen. It has > to do with what each BIOS manufacturer chooses to do with some parts of > memory during start-up. Mostly whether memory remains powered-on through the reset process. Speaking only of i386 / amd64 systems, there are several ways to reboot a computer: - Return to real mode and call the BIOS reboot subroutine. This will perform a cold boot (memory is cleared) or a warm boot (memory should not be cleared, but the BIOS may do so anyway) depending on the value stored at a specific location in memory. - On systems with an i8042 keyboard controller, one of the controller's output lines is usually wired to the CPU's reset pin. Writing a specific value to one of the keyboard controller's control registers causes it to pull that pin low. - Triple fault: set up an empty interrupt descriptor table and trigger an interrupt (e.g. divide by zero). The CPU will fail to invoke the corresponding interrupt descriptor, causing a double fault interrupt, which will fail to execute, causing a CPU reset. The latter two techniques can be used (and *have* been used) to switch back from protected mode to real mode, because they are faster than the documented method. Memory is *not* cleared, and the CPU will start executing whatever code is stored at 0xfffffff0 in real memory; this is normally a far jump into the BIOS reboot subroutine, so if you don't pull any tricks (e.g. jump to your own code that does whatever it was you needed to do in real mode, then return to protected mode), they are equivalent to the first technique. - ACPI reset: store the value specified by RESET_VALUE in the FADT into the register specified by RESET_REG in the FADT. According to the ACPI spec, this results in a cold boot. However, this part of the spec is optional, so not all motherboards support it. FreeBSD will only use ACPI reset if the hw.acpi.handle_reboot sysctl is not zero, which it is by default (mostly, if I recall correctly, because ACPI reset is unreliable). DES -- Dag-Erling Sm?rgrav - des@des.no From koitsu at freebsd.org Thu Nov 20 02:03:15 2008 From: koitsu at freebsd.org (Jeremy Chadwick) Date: Thu Nov 20 02:03:23 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> <20081120063936.GU51761@server.vk2pj.dyndns.org> <20081120070820.GA19307@icarus.home.lan> Message-ID: <20081120100314.GA22639@icarus.home.lan> On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: > On Wed, 19 Nov 2008, Jeremy Chadwick wrote: > >> On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: > >>> I hope that never gets committed - it will make debugging kernel >>> problems much harder. There is already a kern.msgbuf_clear sysctl and >>> maybe people who are concerned about msgbuf leakage need to learn to >>> use it. >> >> And this sysctl is only usable *after* the kernel loads, which means >> you lose all of the messages shown from the time the kernel loads to >> the time the sysctl is set (e.g. hardware detected/configured). This is >> even less acceptable, IMHO. > > But surely you can arrange that the contents are written out to > /var/log/messages first? > > E.g. a sequence like > > - mount /var > - write buffer contents via syslogd > - clear buffer via sysctl > - allow user logins This has two problems, but I'm probably missing something: 1) See my original post, re: users of our systems use "dmesg" to find out what the status of the system is. By "status" I don't mean "from the point the kernel finished to now", I literally mean they *expect* to see the kernel device messages and all that jazz. No, I'm not making this up, nor am I arguing just to hear myself talk (despite popular belief). I can bring these users into the discussion if people feel it would be useful. 2) I don't understand how this would work (meaning, technically and literally: I do not understand). How do messages like "CPU: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz (2992.52-MHz K8-class CPU)" get written to syslog when syslogd isn't even running (or any filesystems) mounted at that time? There must be some magic involved there (since syslog == libc, not syscall) when syslogd starts, but I don't know how it works. > This way the buffer is cleared before any unprivileged users get to do > anything. No kernel changes needed, just a little tweaking of the init > scripts at most. > > If you should have a crash and suspect there is useful data in the > buffer, you can boot to single-user mode (avoiding the clear) and > retrieve it manually. > > Seems like this should make everyone happy. What I'm not understanding is the resistance towards Rink's patch, assuming the tunable defaults to disabled/off. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From koitsu at freebsd.org Thu Nov 20 02:06:03 2008 From: koitsu at freebsd.org (Jeremy Chadwick) Date: Thu Nov 20 02:06:10 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <86r656oha4.fsf@ds4.des.no> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> <20081120063936.GU51761@server.vk2pj.dyndns.org> <20081120070820.GA19307@icarus.home.lan> <86r656oha4.fsf@ds4.des.no> Message-ID: <20081120100600.GB22639@icarus.home.lan> On Thu, Nov 20, 2008 at 10:53:07AM +0100, Dag-Erling Sm?rgrav wrote: > Jeremy Chadwick writes: > > Peter Jeremy writes: > > > This is deliberate. If the system panics, stuff that was in the > > > message buffer (and might not be on disk) can be read when the > > > system reboots. If there is no crashdump, this might be the only > > > record of what happened. > > That doesn't sound deliberate at all -- it sounds like a quirk that > > people (you?) are relying on. > > No, it is deliberate. Just because you don't like it doesn't mean we're > morons. I said nothing about liking/disliking it, nor did I namecall or condescend. Thanks for being a complete prick, des. Jesus christ... -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From neldredge at math.ucsd.edu Thu Nov 20 02:42:41 2008 From: neldredge at math.ucsd.edu (Nate Eldredge) Date: Thu Nov 20 02:42:47 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <20081120100314.GA22639@icarus.home.lan> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> <20081120063936.GU51761@server.vk2pj.dyndns.org> <20081120070820.GA19307@icarus.home.lan> <20081120100314.GA22639@icarus.home.lan> Message-ID: On Thu, 20 Nov 2008, Jeremy Chadwick wrote: > On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: >> On Wed, 19 Nov 2008, Jeremy Chadwick wrote: >> >>> On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: >> >>>> I hope that never gets committed - it will make debugging kernel >>>> problems much harder. There is already a kern.msgbuf_clear sysctl and >>>> maybe people who are concerned about msgbuf leakage need to learn to >>>> use it. >>> >>> And this sysctl is only usable *after* the kernel loads, which means >>> you lose all of the messages shown from the time the kernel loads to >>> the time the sysctl is set (e.g. hardware detected/configured). This is >>> even less acceptable, IMHO. >> >> But surely you can arrange that the contents are written out to >> /var/log/messages first? >> >> E.g. a sequence like >> >> - mount /var >> - write buffer contents via syslogd >> - clear buffer via sysctl >> - allow user logins > > This has two problems, but I'm probably missing something: > > 1) See my original post, re: users of our systems use "dmesg" to find > out what the status of the system is. By "status" I don't mean "from > the point the kernel finished to now", I literally mean they *expect* > to see the kernel device messages and all that jazz. No, I'm not > making this up, nor am I arguing just to hear myself talk (despite > popular belief). I can bring these users into the discussion if people > feel it would be useful. I forgot about that point. I can sympathize with those users; I feel the same way. It's a good way to learn about a system as a mere user (since usually sysadmins don't remember or bother to disable it). However, in my experience dmesg isn't really the best thing for that purpose; the kernel message buffer tends to get wiped out once the system has been up for a while. (It fills with ipfw logs, ethernet link state changes, etc.) Maybe a better approach would be to point them to /var/log/messages or whichever log file stores them permanently. Or, better yet, do some syslogd magic to make a logfile that can be appropriately readable but doesn't have any overly sensitive messages directed there (e.g. kernel yes, sshd no). > 2) I don't understand how this would work (meaning, technically and > literally: I do not understand). How do messages like "CPU: Intel(R) > Core(TM)2 Duo CPU E8400 @ 3.00GHz (2992.52-MHz K8-class CPU)" get > written to syslog when syslogd isn't even running (or any filesystems) > mounted at that time? There must be some magic involved there (since > syslog == libc, not syscall) when syslogd starts, but I don't know > how it works. I think you're conflating a couple of things, and I also explained my idea poorly. As I understand it (from memory, which is a little vague), syslogd gets messages from two places: from the kernel via /dev/klog, and from other processes via a Unix domain socket in /var/run. These messages then get sent to the appropriate log files. The syslog(3) function of libc just connects and writes the message to the Unix domain socket. If syslogd isn't running to listen on that socket, syslog(3) won't work very well. Now /dev/klog should be a direct line to the kernel's message buffer. So when syslogd starts and reads /dev/klog for the first time, it will get everything that's accumulated so far, including the earliest boot messages. It should then write them to the appropriate log files. This already works, which is why /var/log/messages contains the kernel copyright message, etc. My idea is, after syslogd does this, but before the system goes multi-user, you should clear the kernel buffer. Early boot messages are already in the log files, so they won't be lost. Maybe the best thing would be to build this functionality into syslogd itself, to minimize the possibility of losing messages due to a race. >> This way the buffer is cleared before any unprivileged users get to do >> anything. No kernel changes needed, just a little tweaking of the init >> scripts at most. >> >> If you should have a crash and suspect there is useful data in the >> buffer, you can boot to single-user mode (avoiding the clear) and >> retrieve it manually. >> >> Seems like this should make everyone happy. > > What I'm not understanding is the resistance towards Rink's patch, > assuming the tunable defaults to disabled/off. It seems reasonable to me. The only catch I can see is that if you have a crash and you want to look at the message buffer after rebooting, you'll have to remember to stop at the loader prompt and turn off that tunable. Which might be easy to forget in the heat of the moment. -- Nate Eldredge neldredge@math.ucsd.edu From caelian at gmail.com Thu Nov 20 05:18:26 2008 From: caelian at gmail.com (Pascal Hofstee) Date: Thu Nov 20 05:18:32 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: References: Message-ID: <20081120134836.2870a827@nebuchadnezzar> On Thu, 20 Nov 2008 01:46:31 -0000 "Pegasus Mc Cleaft" wrote: > Hi everyone, > > I am having difficulties rebuilding the world after some patches > were made today. I was wondering if anyone else is experiencing the > same troubles? > > ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x0 -o > gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/lib/crt0.o > zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No such > file or directory *** Error code 1 > > Stop in /usr/src/sys/boot/i386/gptzfsboot. > *** Error code 1 I am experiencing the exact same problem with a fresh svn checkout -- Pascal Hofstee From keramida at ceid.upatras.gr Thu Nov 20 06:17:07 2008 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Thu Nov 20 06:17:13 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? In-Reply-To: <200811191724.mAJHOL8r062364@fire.js.berklix.net> (Julian Stacey's message of "Wed, 19 Nov 2008 18:24:21 +0100") References: <200811191724.mAJHOL8r062364@fire.js.berklix.net> Message-ID: <8763miy3jk.fsf@kobe.laptop> On Wed, 19 Nov 2008 18:24:21 +0100, "Julian Stacey" wrote: > Hi hackers, > Maybe Some of you might suggest some software I might install, Wiki I guess. ? > I got zero response from ports@, I could use some reccomendations please. > PS From http://wiki.freebsd.org/HelpContents I tried > cd /usr/ports/www ; vi *iki*/pkg-descr > or is /usr/ports/www/moinmoin the way to go ? > Thanks. > ----------- > > Subject: Reccomendation for ports for web based club events > forthcoming diary ? > > Can anyone reccomend some ports to install on a FreeBSD web server, > for a club of mostly non technical people, to support: > - All club members can add events to a forthcoming calendar, > - All club members can request server to prepare a listing > of next next upcoming events, to download (probably in PDF, > or perhaps tbl to a pipe or ? > - A list of moderators can delete fake events from robots & the malicious. > - Preferably moderators should not themselves be capable of > deleting logged event submission, but only capable of deleting > events formatted to the ouput printable programme sheet. (To > autopsy for suspect rogue moderators) > - I guess first entry criteria might be a fuzzy picture for human > to decode password from). 2nd might be mail return for confirm password, > - & 3rd, A majordomo (later mailman) maintained list of club members & > moderators etc is available for automated validation. > - I hope there will be some packages available, > http & probably wiki based etc, that will come close enough ? Hi Julian, I have been working with `OddMuse' in the EmacsWiki[1] as a user and as an admin/moderator in some installations of my own. The "UI" of the wiki is pretty simple, and it does have a very low level of requirements for becoming a `wiki editor', so it may be a good choice for a Wiki where non-technical people produce a lot of the content. OddMuse does have a relatively _unique_ way of treating Wiki content and may require a bit of Perl hackery to configure captchas, but I like the fact that it is basically a relatively small `wiki core' that is fairly trivial to extend by writing Perl modules. See for more details about the OddMuse wiki engine. Wikipedia has a list of other Wiki software at http://en.wikipedia.org/wiki/List_of_wiki_software that may also be useful. From kevin at your.org Thu Nov 20 07:39:01 2008 From: kevin at your.org (Kevin Day) Date: Thu Nov 20 07:39:32 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <61AFAC99-50A3-40BC-8EF0-809FB78C3B49@dragondata.com> References: <20081028081154.GQ6808@hoeg.nl> <20081118213410.GA81783@hoeg.nl> <20081118214919.GM83287@bunrab.catwhisker.org> <7d6fde3d0811190202p4f6d8941h3932b70b8fe1a93a@mail.gmail.com> <20081119104731.GA83366@icarus.home.lan> <20081120063936.GU51761@server.vk2pj.dyndns.org> <20081120070820.GA19307@icarus.home.lan> <20081120100314.GA22639@icarus.home.lan> <61AFAC99-50A3-40BC-8EF0-809FB78C3B49@dragondata.com> Message-ID: On Nov 20, 2008, at 4:03 AM, Jeremy Chadwick wrote: > > This has two problems, but I'm probably missing something: > > 1) See my original post, re: users of our systems use "dmesg" to find > out what the status of the system is. By "status" I don't mean "from > the point the kernel finished to now", I literally mean they *expect* > to see the kernel device messages and all that jazz. No, I'm not > making this up, nor am I arguing just to hear myself talk (despite > popular belief). I can bring these users into the discussion if > people > feel it would be useful. Sorry for jumping in late, but... cat /var/run/dmesg.boot Is that acceptable? I know it depends on what end users want, but some of my old hosting customers really just wanted to see the specs of the box and nothing else. Making dmesg a shell script that just cats that file satisfied everyone who asked. Also: > On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: >> On 2008-Nov-19 02:47:31 -0800, Jeremy Chadwick >> wrote: >>> There's a known "issue" with the kernel message buffer though: >>> it's not >>> NULL'd out upon reboot. >> >> This is deliberate. If the system panics, stuff that was in the >> message buffer (and might not be on disk) can be read when the system >> reboots. If there is no crashdump, this might be the only record of >> what happened. > > That doesn't sound deliberate at all -- it sounds like a quirk that > people (you?) are relying on. I do not think any piece of the FreeBSD > system (e.g. savecore, etc.) relies on this behaviour. > > You're under the mentality that the information is *always* available > after a panic/reboot -- it isn't. I have 4 different Supermicro > motherboards (all from different years) which will "most of the time" > lose the msgbuf after rebooting from single-user -- but sometimes the > msgbuf is retained. And no, bad hardware is not responsible for the > randomness of the problem. > > I think it's been discussed in the past how/why this can happen. It > has > to do with what each BIOS manufacturer chooses to do with some parts > of > memory during start-up. I'm sure the "Quick Boot" (e.g. no extensive > memory test, which really doesn't test anything these days) option > plays > a role, and that option is enabled by default on all motherboards I've > used in the past 10 years. I've been involved with a few embedded systems, some BSD based some not. In a few cases we've used custom BIOSes on the motherboard. At least one BIOS SDK specifically describes this as a feature. What is exposed to the end user as "Quick Boot" is actually several options that the motherboard designer/BIOS configurer can select. One of which is specifying which chunks of memory should be preserved after a reboot, up to "memory tests should be as non-destructive as possible". While you can't *rely* on it, with careful use of atomic writes and state checking you can pick up where you left off after a reboot on an embedded device that has no long-term storage if it was a warm boot. Or, gathering crash dumps and sending them off to the network. Here, the dmesg buffer is a simple ring buffer in the kernel. The start/end pointers and contents of the ring buffer are deliberately not cleared after a reboot in FreeBSD to at least make the information available if the BIOS didn't clobber it. This can be extremely useful in those "a box on the other side of the world keeps rebooting with no panic", if you can tell the BIOS to skip the memory check. In one case I even (ab)used some extra video ram on the motherboard as a dmesg buffer since the video bios didn't wipe it on boot, but the system's bios insisted on doing a full memory test each time. If you have a BIOS that's sometimes but not always wiping the buffer, it's probably because a few bits are being lost while the motherboard turns the DRAM refresh off during the reboot. Some DIMMS can handle no refresh for tens of seconds, some start popping bit errors rather quickly. Take a look at sys/kern/subr_msgbuf.c:msgbuf_reinit. There's a magic number it looks for as well as a checksum on the whole buffer that's updated after every write. If anything changes, the kernel throws out the whole buffer and starts fresh. If you boot with "-v" you'll probably see "msgbuf cksum mismatch". While I don't think any of the shipping FreeBSD tools rely on this behavior, I know I tried submitting a patch back in the FreeBSD-2.2(?) days to fix that "overlooked uninitialized buffer" and got educated by jkh pretty quickly. :) From fjwcash at gmail.com Thu Nov 20 09:00:11 2008 From: fjwcash at gmail.com (Freddie Cash) Date: Thu Nov 20 09:00:18 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: References: <20081120100314.GA22639@icarus.home.lan> Message-ID: <200811200831.47062.fjwcash@gmail.com> On November 20, 2008 02:42 am Nate Eldredge wrote: > On Thu, 20 Nov 2008, Jeremy Chadwick wrote: > > On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: > >> On Wed, 19 Nov 2008, Jeremy Chadwick wrote: > >>> On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: > >>>> I hope that never gets committed - it will make debugging kernel > >>>> problems much harder. There is already a kern.msgbuf_clear sysctl > >>>> and maybe people who are concerned about msgbuf leakage need to > >>>> learn to use it. > >>> > >>> And this sysctl is only usable *after* the kernel loads, which > >>> means you lose all of the messages shown from the time the kernel > >>> loads to the time the sysctl is set (e.g. hardware > >>> detected/configured). This is even less acceptable, IMHO. > >> > >> But surely you can arrange that the contents are written out to > >> /var/log/messages first? > >> > >> E.g. a sequence like > >> > >> - mount /var > >> - write buffer contents via syslogd > >> - clear buffer via sysctl > >> - allow user logins > > > > This has two problems, but I'm probably missing something: > > > > 1) See my original post, re: users of our systems use "dmesg" to find > > out what the status of the system is. By "status" I don't mean "from > > the point the kernel finished to now", I literally mean they *expect* > > to see the kernel device messages and all that jazz. No, I'm not > > making this up, nor am I arguing just to hear myself talk (despite > > popular belief). I can bring these users into the discussion if > > people feel it would be useful. > > I forgot about that point. I can sympathize with those users; I > feel the same way. It's a good way to learn about a system as a > mere user (since usually sysadmins don't remember or bother to > disable it). > > However, in my experience dmesg isn't really the best thing for that > purpose; the kernel message buffer tends to get wiped out once the > system has been up for a while. (It fills with ipfw logs, ethernet > link state changes, etc.) > > Maybe a better approach would be to point them to /var/log/messages > or whichever log file stores them permanently. I think what you are looking for is /var/run/dmesg.boot, which stores just the dmesg info from the initial boot. Nothing gets logged to this after the boot is complete. This file has been a life saver quite a few times since I discovered it, and is something I really miss when working with mis-behaving Linux systems. -- Freddie Cash fjwcash@gmail.com From jhs at berklix.org Thu Nov 20 09:44:12 2008 From: jhs at berklix.org (Julian Stacey) Date: Thu Nov 20 09:44:19 2008 Subject: Reccomendation for tools to use on FreeBSD for a wiki ? In-Reply-To: Your message "Thu, 20 Nov 2008 09:32:33 +0200." Message-ID: <200811201743.mAKHhOE4012470@fire.js.berklix.net> Hi, > Will you tell us which one you selected? Happy to, when I know which & why, likely it'll be a while before that. Just an unpaid hobby, not spending 8 hours/day on selection/install tuning. Whether then I'll remember to post, or you'll want to remind me, we'll see :-) PS I started with /usr/ports/www/moinmoin as that's what wiki.freebs.org uses, & then I seitched to CGI not FCGI option as it sounded easier. Then I saw post by Giogios & need to think about that. Thanks Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From olivier at gid0.org Thu Nov 20 10:57:55 2008 From: olivier at gid0.org (Olivier SMEDTS) Date: Thu Nov 20 10:58:02 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> References: <20081120134836.2870a827@nebuchadnezzar> <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> Message-ID: <367b2c980811201057y4e44e0ehb227d37702d98c43@mail.gmail.com> What is your MACHINE_ARCH ? Mine is amd64, I think there's a problem with the conditional in sys/boot/i386/gptzfsboot/Makefile. ld doesn't need gptzfsboot.o on i386. Olivier 2008/11/20 Olivier SMEDTS : > 2008/11/20 Pascal Hofstee : >> On Thu, 20 Nov 2008 01:46:31 -0000 >> "Pegasus Mc Cleaft" wrote: >> >>> Hi everyone, >>> >>> I am having difficulties rebuilding the world after some patches >>> were made today. I was wondering if anyone else is experiencing the >>> same troubles? >>> >>> ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x0 -o >>> gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/lib/crt0.o >>> zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No such >>> file or directory *** Error code 1 >>> >>> Stop in /usr/src/sys/boot/i386/gptzfsboot. >>> *** Error code 1 >> >> I am experiencing the exact same problem with a fresh svn checkout > > Just my "me too". > I did not experience the problem 24 hours ago (after ZFS version 13 > update and zfsboot import). > >> >> -- >> Pascal Hofstee >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> > > > > -- > Olivier Smedts _ > ASCII ribbon campaign ( ) > e-mail: olivier@gid0.org - against HTML email & vCards X > www: http://www.gid0.org - against proprietary attachments / \ > > "Il y a seulement 10 sortes de gens dans le monde : > ceux qui comprennent le binaire, > et ceux qui ne le comprennent pas." > -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." From olivier at gid0.org Thu Nov 20 11:05:14 2008 From: olivier at gid0.org (Olivier SMEDTS) Date: Thu Nov 20 11:05:21 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <20081120134836.2870a827@nebuchadnezzar> References: <20081120134836.2870a827@nebuchadnezzar> Message-ID: <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> 2008/11/20 Pascal Hofstee : > On Thu, 20 Nov 2008 01:46:31 -0000 > "Pegasus Mc Cleaft" wrote: > >> Hi everyone, >> >> I am having difficulties rebuilding the world after some patches >> were made today. I was wondering if anyone else is experiencing the >> same troubles? >> >> ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x0 -o >> gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/lib/crt0.o >> zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No such >> file or directory *** Error code 1 >> >> Stop in /usr/src/sys/boot/i386/gptzfsboot. >> *** Error code 1 > > I am experiencing the exact same problem with a fresh svn checkout Just my "me too". I did not experience the problem 24 hours ago (after ZFS version 13 update and zfsboot import). > > -- > Pascal Hofstee > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." From olivier at gid0.org Thu Nov 20 11:07:09 2008 From: olivier at gid0.org (Olivier SMEDTS) Date: Thu Nov 20 11:07:22 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> References: <20081120134836.2870a827@nebuchadnezzar> <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> Message-ID: <367b2c980811201107x7fc859b8yeee0816a37eae470@mail.gmail.com> 2008/11/20 Olivier SMEDTS : > 2008/11/20 Pascal Hofstee : >> On Thu, 20 Nov 2008 01:46:31 -0000 >> "Pegasus Mc Cleaft" wrote: >> >>> Hi everyone, >>> >>> I am having difficulties rebuilding the world after some patches >>> were made today. I was wondering if anyone else is experiencing the >>> same troubles? >>> >>> ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x0 -o >>> gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/lib/crt0.o >>> zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No such >>> file or directory *** Error code 1 >>> >>> Stop in /usr/src/sys/boot/i386/gptzfsboot. >>> *** Error code 1 >> >> I am experiencing the exact same problem with a fresh svn checkout > > Just my "me too". > I did not experience the problem 24 hours ago (after ZFS version 13 > update and zfsboot import). That's it. Seems to work with the following patch : --- sys/boot/i386/gptzfsboot/Makefile.orig 2008-11-20 19:58:45.000000000 +0100 +++ sys/boot/i386/gptzfsboot/Makefile 2008-11-20 20:01:53.000000000 +0100 @@ -65,7 +65,7 @@ zfsboot.o: ${.CURDIR}/../../zfs/zfsimpl.c .if ${MACHINE_ARCH} == "amd64" -beforedepend gptzfsboot.o: machine +beforedepend gptzfsboot.bin: machine CLEANFILES+= machine machine: ln -sf ${.CURDIR}/../../../i386/include machine I've cc'ed current@ because HEAD is broken on amd64 for now. Cheers, Olivier > >> >> -- >> Pascal Hofstee >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> > > > > -- > Olivier Smedts _ > ASCII ribbon campaign ( ) > e-mail: olivier@gid0.org - against HTML email & vCards X > www: http://www.gid0.org - against proprietary attachments / \ > > "Il y a seulement 10 sortes de gens dans le monde : > ceux qui comprennent le binaire, > et ceux qui ne le comprennent pas." > -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." From olivier at gid0.org Thu Nov 20 11:41:15 2008 From: olivier at gid0.org (Olivier SMEDTS) Date: Thu Nov 20 11:41:22 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <367b2c980811201107x7fc859b8yeee0816a37eae470@mail.gmail.com> References: <20081120134836.2870a827@nebuchadnezzar> <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> <367b2c980811201107x7fc859b8yeee0816a37eae470@mail.gmail.com> Message-ID: <367b2c980811201141j42977204ne6052000a0d095ab@mail.gmail.com> 2008/11/20 Olivier SMEDTS : > 2008/11/20 Olivier SMEDTS : >> 2008/11/20 Pascal Hofstee : >>> On Thu, 20 Nov 2008 01:46:31 -0000 >>> "Pegasus Mc Cleaft" wrote: >>> >>>> Hi everyone, >>>> >>>> I am having difficulties rebuilding the world after some patches >>>> were made today. I was wondering if anyone else is experiencing the >>>> same troubles? >>>> >>>> ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x0 -o >>>> gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/lib/crt0.o >>>> zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No such >>>> file or directory *** Error code 1 >>>> >>>> Stop in /usr/src/sys/boot/i386/gptzfsboot. >>>> *** Error code 1 >>> >>> I am experiencing the exact same problem with a fresh svn checkout >> >> Just my "me too". >> I did not experience the problem 24 hours ago (after ZFS version 13 >> update and zfsboot import). > > That's it. Seems to work with the following patch : > > --- sys/boot/i386/gptzfsboot/Makefile.orig 2008-11-20 > 19:58:45.000000000 +0100 > +++ sys/boot/i386/gptzfsboot/Makefile 2008-11-20 20:01:53.000000000 +0100 > @@ -65,7 +65,7 @@ > zfsboot.o: ${.CURDIR}/../../zfs/zfsimpl.c > > .if ${MACHINE_ARCH} == "amd64" > -beforedepend gptzfsboot.o: machine > +beforedepend gptzfsboot.bin: machine > CLEANFILES+= machine > machine: > ln -sf ${.CURDIR}/../../../i386/include machine Sorry for replying again to my own post :) The patch is crap, in fact it just breaks the already broken conditional. At least I can buildworld on amd64 now (I don't use the recently introduced gptzfsboot). Makefile experts ? > > I've cc'ed current@ because HEAD is broken on amd64 for now. > > Cheers, > > Olivier > > >> >>> >>> -- >>> Pascal Hofstee >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >>> >> >> >> >> -- >> Olivier Smedts _ >> ASCII ribbon campaign ( ) >> e-mail: olivier@gid0.org - against HTML email & vCards X >> www: http://www.gid0.org - against proprietary attachments / \ >> >> "Il y a seulement 10 sortes de gens dans le monde : >> ceux qui comprennent le binaire, >> et ceux qui ne le comprennent pas." >> > > > > -- > Olivier Smedts _ > ASCII ribbon campaign ( ) > e-mail: olivier@gid0.org - against HTML email & vCards X > www: http://www.gid0.org - against proprietary attachments / \ > > "Il y a seulement 10 sortes de gens dans le monde : > ceux qui comprennent le binaire, > et ceux qui ne le comprennent pas." > -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." From ken at mthelicon.com Thu Nov 20 11:49:09 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Thu Nov 20 11:49:15 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <367b2c980811201057y4e44e0ehb227d37702d98c43@mail.gmail.com> References: <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> <367b2c980811201057y4e44e0ehb227d37702d98c43@mail.gmail.com> Message-ID: <200811201949.06286.ken@mthelicon.com> On Thursday 20 November 2008 18:57:53 Olivier SMEDTS wrote: > What is your MACHINE_ARCH ? > Mine is amd64, I think there's a problem with the conditional in > sys/boot/i386/gptzfsboot/Makefile. > ld doesn't need gptzfsboot.o on i386. > > Olivier > > 2008/11/20 Olivier SMEDTS : > > 2008/11/20 Pascal Hofstee : > >> On Thu, 20 Nov 2008 01:46:31 -0000 > >> > >> "Pegasus Mc Cleaft" wrote: > >>> Hi everyone, > >>> > >>> I am having difficulties rebuilding the world after some patches > >>> were made today. I was wondering if anyone else is experiencing the > >>> same troubles? Hi Oliver, My machine is an Core2 Quad running under AMD64. (CPUTYPE?=core2) Thanks for replying. It puts my mind to ease because I was thinking it was a problem I created (I recently moved the /usr/src directory into a seperate zfs filing system) Peg From olivier at gid0.org Thu Nov 20 12:36:15 2008 From: olivier at gid0.org (Olivier SMEDTS) Date: Thu Nov 20 12:36:22 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <200811201949.06286.ken@mthelicon.com> References: <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> <367b2c980811201057y4e44e0ehb227d37702d98c43@mail.gmail.com> <200811201949.06286.ken@mthelicon.com> Message-ID: <20081120201336.GA66837@q.gid0.org> On Thu, Nov 20, 2008 at 07:49:06PM +0000, Pegasus Mc Cleaft wrote: > On Thursday 20 November 2008 18:57:53 Olivier SMEDTS wrote: > > What is your MACHINE_ARCH ? > > Mine is amd64, I think there's a problem with the conditional in > > sys/boot/i386/gptzfsboot/Makefile. > > ld doesn't need gptzfsboot.o on i386. Now I think I've got it : All the '.if ${MACHINE_ARCH} == "amd64"' which replace the amd64 machine link with an i386 one are useless on 7.0 and -CURRENT since rev. 1.17 of sys/boot/efi/libefi/Makefile. This file already takes care of replacing MACHINE_ARCH. And I don't think zfs*boot will be in 6-STABLE. You can apply the following patch in sys/boot/i386. I'll submit a PR if it's not committed before. Cheers, Olivier > > > > Olivier > > > > 2008/11/20 Olivier SMEDTS : > > > 2008/11/20 Pascal Hofstee : > > >> On Thu, 20 Nov 2008 01:46:31 -0000 > > >> > > >> "Pegasus Mc Cleaft" wrote: > > >>> Hi everyone, > > >>> > > >>> I am having difficulties rebuilding the world after some patches > > >>> were made today. I was wondering if anyone else is experiencing the > > >>> same troubles? > > > > Hi Oliver, > My machine is an Core2 Quad running under AMD64. (CPUTYPE?=core2) > > Thanks for replying. It puts my mind to ease because I was thinking it was a > problem I created (I recently moved the /usr/src directory into a seperate zfs > filing system) > > Peg -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." -------------- next part -------------- --- boot2/Makefile.orig 2008-11-20 20:56:31.000000000 +0100 +++ boot2/Makefile 2008-11-20 20:56:42.000000000 +0100 @@ -94,11 +94,4 @@ ORG1=`printf "%d" ${ORG1}` \ REL1=`printf "%d" ${REL1}` > ${.TARGET} -.if ${MACHINE_ARCH} == "amd64" -beforedepend boot2.s: machine -CLEANFILES+= machine -machine: - ln -sf ${.CURDIR}/../../../i386/include machine -.endif - .include --- gptboot/Makefile.orig 2008-11-20 20:50:34.000000000 +0100 +++ gptboot/Makefile 2008-11-20 20:50:40.000000000 +0100 @@ -67,11 +67,4 @@ gptboot.o: ${.CURDIR}/../../common/ufsread.c -.if ${MACHINE_ARCH} == "amd64" -beforedepend gptboot.o: machine -CLEANFILES+= machine -machine: - ln -sf ${.CURDIR}/../../../i386/include machine -.endif - .include --- libfirewire/Makefile.orig 2008-11-20 20:56:07.000000000 +0100 +++ libfirewire/Makefile 2008-11-20 20:56:18.000000000 +0100 @@ -16,15 +16,4 @@ CFLAGS+= -Wformat -Wall -.if ${MACHINE_ARCH} == "amd64" -CLEANFILES+= machine -machine: - ln -sf ${.CURDIR}/../../../i386/include machine -.endif - .include - -.if ${MACHINE_ARCH} == "amd64" -beforedepend ${OBJS}: machine -.endif - --- libi386/Makefile.orig 2008-11-20 20:55:38.000000000 +0100 +++ libi386/Makefile 2008-11-20 20:55:55.000000000 +0100 @@ -45,14 +45,4 @@ # the location of libstand CFLAGS+= -I${.CURDIR}/../../../../lib/libstand/ -.if ${MACHINE_ARCH} == "amd64" -CLEANFILES+= machine -machine: - ln -sf ${.CURDIR}/../../../i386/include machine -.endif - .include - -.if ${MACHINE_ARCH} == "amd64" -beforedepend ${OBJS}: machine -.endif --- loader/Makefile.orig 2008-11-20 20:54:43.000000000 +0100 +++ loader/Makefile 2008-11-20 20:54:58.000000000 +0100 @@ -110,10 +110,3 @@ LDADD= ${LIBFICL} ${LIBFIREWIRE} ${LIBZFS} ${LIBI386} -lstand .include - -.if ${MACHINE_ARCH} == "amd64" -beforedepend ${OBJS}: machine -CLEANFILES+= machine -machine: - ln -sf ${.CURDIR}/../../../i386/include machine -.endif --- zfsboot/Makefile.orig 2008-11-20 20:54:18.000000000 +0100 +++ zfsboot/Makefile 2008-11-20 20:54:27.000000000 +0100 @@ -98,11 +98,4 @@ ORG1=`printf "%d" ${ORG1}` \ REL1=`printf "%d" ${REL1}` > ${.TARGET} -.if ${MACHINE_ARCH} == "amd64" -beforedepend zfsboot.s: machine -CLEANFILES+= machine -machine: - ln -sf ${.CURDIR}/../../../i386/include machine -.endif - .include --- gptzfsboot/Makefile.orig 2008-11-20 19:58:45.000000000 +0100 +++ gptzfsboot/Makefile 2008-11-20 20:50:25.000000000 +0100 @@ -64,11 +64,4 @@ zfsboot.o: ${.CURDIR}/../../zfs/zfsimpl.c -.if ${MACHINE_ARCH} == "amd64" -beforedepend gptzfsboot.o: machine -CLEANFILES+= machine -machine: - ln -sf ${.CURDIR}/../../../i386/include machine -.endif - .include From olivier at gid0.org Thu Nov 20 13:52:01 2008 From: olivier at gid0.org (Olivier SMEDTS) Date: Thu Nov 20 13:52:13 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <20081120201336.GA66837@q.gid0.org> References: <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> <367b2c980811201057y4e44e0ehb227d37702d98c43@mail.gmail.com> <200811201949.06286.ken@mthelicon.com> <20081120201336.GA66837@q.gid0.org> Message-ID: <367b2c980811201351q3e920fdex5b377ceb8dc0fad3@mail.gmail.com> 2008/11/20 Olivier SMEDTS : > On Thu, Nov 20, 2008 at 07:49:06PM +0000, Pegasus Mc Cleaft wrote: >> On Thursday 20 November 2008 18:57:53 Olivier SMEDTS wrote: >> > What is your MACHINE_ARCH ? >> > Mine is amd64, I think there's a problem with the conditional in >> > sys/boot/i386/gptzfsboot/Makefile. >> > ld doesn't need gptzfsboot.o on i386. > > Now I think I've got it : > > All the '.if ${MACHINE_ARCH} == "amd64"' which replace the amd64 machine > link with an i386 one are useless on 7.0 and -CURRENT since rev. 1.17 of > sys/boot/efi/libefi/Makefile. This file already takes care of replacing > MACHINE_ARCH. And I don't think zfs*boot will be in 6-STABLE. Wow, still not good... I was too enthusiastic while waiting for a fresh buildworld to finish. It worked without cleaning though (buildworld without patch then patch then make clean in sys/boot/i386 then finish buildworld without cleaning). Must have missed something. I give up for today, I think I really must sleep :) > > You can apply the following patch in sys/boot/i386. I'll submit a PR if > it's not committed before. > > Cheers, > Olivier > > >> > >> > Olivier >> > >> > 2008/11/20 Olivier SMEDTS : >> > > 2008/11/20 Pascal Hofstee : >> > >> On Thu, 20 Nov 2008 01:46:31 -0000 >> > >> >> > >> "Pegasus Mc Cleaft" wrote: >> > >>> Hi everyone, >> > >>> >> > >>> I am having difficulties rebuilding the world after some patches >> > >>> were made today. I was wondering if anyone else is experiencing the >> > >>> same troubles? >> >> >> >> Hi Oliver, >> My machine is an Core2 Quad running under AMD64. (CPUTYPE?=core2) >> >> Thanks for replying. It puts my mind to ease because I was thinking it was a >> problem I created (I recently moved the /usr/src directory into a seperate zfs >> filing system) >> >> Peg -- Olivier Smedts _ ASCII ribbon campaign ( ) e-mail: olivier@gid0.org - against HTML email & vCards X www: http://www.gid0.org - against proprietary attachments / \ "Il y a seulement 10 sortes de gens dans le monde : ceux qui comprennent le binaire, et ceux qui ne le comprennent pas." From yanefbsd at gmail.com Fri Nov 21 17:54:26 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Fri Nov 21 17:54:32 2008 Subject: [Testers wanted] /dev/console cleanups In-Reply-To: <200811200831.47062.fjwcash@gmail.com> References: <20081120100314.GA22639@icarus.home.lan> <200811200831.47062.fjwcash@gmail.com> Message-ID: <7d6fde3d0811211754s220cc980l1a43109155c7013f@mail.gmail.com> On Thu, Nov 20, 2008 at 8:31 AM, Freddie Cash wrote: > On November 20, 2008 02:42 am Nate Eldredge wrote: >> On Thu, 20 Nov 2008, Jeremy Chadwick wrote: >> > On Wed, Nov 19, 2008 at 11:48:36PM -0800, Nate Eldredge wrote: >> >> On Wed, 19 Nov 2008, Jeremy Chadwick wrote: >> >>> On Thu, Nov 20, 2008 at 05:39:36PM +1100, Peter Jeremy wrote: >> >>>> I hope that never gets committed - it will make debugging kernel >> >>>> problems much harder. There is already a kern.msgbuf_clear sysctl >> >>>> and maybe people who are concerned about msgbuf leakage need to >> >>>> learn to use it. >> >>> >> >>> And this sysctl is only usable *after* the kernel loads, which >> >>> means you lose all of the messages shown from the time the kernel >> >>> loads to the time the sysctl is set (e.g. hardware >> >>> detected/configured). This is even less acceptable, IMHO. >> >> >> >> But surely you can arrange that the contents are written out to >> >> /var/log/messages first? >> >> >> >> E.g. a sequence like >> >> >> >> - mount /var >> >> - write buffer contents via syslogd >> >> - clear buffer via sysctl >> >> - allow user logins >> > >> > This has two problems, but I'm probably missing something: >> > >> > 1) See my original post, re: users of our systems use "dmesg" to find >> > out what the status of the system is. By "status" I don't mean "from >> > the point the kernel finished to now", I literally mean they *expect* >> > to see the kernel device messages and all that jazz. No, I'm not >> > making this up, nor am I arguing just to hear myself talk (despite >> > popular belief). I can bring these users into the discussion if >> > people feel it would be useful. >> >> I forgot about that point. I can sympathize with those users; I >> feel the same way. It's a good way to learn about a system as a >> mere user (since usually sysadmins don't remember or bother to >> disable it). >> >> However, in my experience dmesg isn't really the best thing for that >> purpose; the kernel message buffer tends to get wiped out once the >> system has been up for a while. (It fills with ipfw logs, ethernet >> link state changes, etc.) >> >> Maybe a better approach would be to point them to /var/log/messages >> or whichever log file stores them permanently. > > I think what you are looking for is /var/run/dmesg.boot, which stores just > the dmesg info from the initial boot. Nothing gets logged to this after > the boot is complete. This file has been a life saver quite a few times > since I discovered it, and is something I really miss when working with > mis-behaving Linux systems. > > -- > Freddie Cash > fjwcash@gmail.com [gcooper@optimus ~]$ grep -rs dmesg.boot /usr/src/ /usr/src/contrib/ntp/scripts/freq_adj.in: open(DM, "/var/run/dmesg.boot"); /usr/src/etc/defaults/rc.conf:dmesg_enable="YES" # Save dmesg(8) to /var/run/dmesg.boot /usr/src/etc/rc.d/dmesg:dmesg_file="/var/run/dmesg.boot" /usr/src/sbin/dmesg/dmesg.8:.Bl -tag -width ".Pa /var/run/dmesg.boot" -compact /usr/src/sbin/dmesg/dmesg.8:.It Pa /var/run/dmesg.boot [gcooper@optimus ~]$ /etc/rc.d/dmesg does this. -Garrett From kuku at kukulies.org Sat Nov 22 03:01:19 2008 From: kuku at kukulies.org (Christoph Kukulies) Date: Sat Nov 22 03:01:27 2008 Subject: mounting Mac OS .dmg files? Message-ID: <4927E659.3010602@kukulies.org> Since the underlying OS of Mac OS is FreeBSD I'm wondering whether itr is possible to mount a .dmg file as it is used in Apple software distribution for the Mac OS. I would like to install an Apple iPhone configuration utility on my FreeBSD box (iPhoneConfigurationUtility.dmg). It consists of a web application that is supposedly capable of doing updates of Apple iPhones through a web service. (good for company wide distribution of address book information and other data that iPhones can hold). -- Christoph Kukulies From onemda at gmail.com Sat Nov 22 04:55:40 2008 From: onemda at gmail.com (Paul B. Mahol) Date: Sat Nov 22 04:55:46 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <4927E659.3010602@kukulies.org> References: <4927E659.3010602@kukulies.org> Message-ID: <3a142e750811220455u1fe2cc48sb25a1911a6bfe351@mail.gmail.com> On 11/22/08, Christoph Kukulies wrote: > Since the underlying OS of Mac OS is FreeBSD Wrong. > I'm wondering whether itr is possible to mount a > .dmg file as it is used in Apple software distribution for the Mac OS. I > would like to install an > Apple iPhone configuration utility on my FreeBSD box > (iPhoneConfigurationUtility.dmg). It consists of a > web application that is supposedly capable of doing updates of Apple > iPhones through a web service. > (good for company wide distribution of address book information and > other data that iPhones can hold). Currently not always possible - depends which actual fs is stored in .dmg file. Some time ago there was utility in development which allowed mounting of hfs+ on FreeBSD. There is dmg2iso (but it is not in ports ....) -- Paul From des at des.no Sat Nov 22 05:58:17 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Sat Nov 22 05:58:24 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <4927E659.3010602@kukulies.org> (Christoph Kukulies's message of "Sat, 22 Nov 2008 12:00:41 +0100") References: <4927E659.3010602@kukulies.org> Message-ID: <863ahjc16w.fsf@ds4.des.no> Christoph Kukulies writes: > Since the underlying OS of Mac OS is FreeBSD It isn't, as someone else pointed out, and hence... > I'm wondering whether itr is possible to mount a .dmg file as it is > used in Apple software distribution for the Mac OS. I would like to > install an Apple iPhone configuration utility on my FreeBSD box > (iPhoneConfigurationUtility.dmg). ...Mac OS applications won't run on FreeBSD, so there's no point in even trying. DES -- Dag-Erling Sm?rgrav - des@des.no From kuku at kukulies.org Sat Nov 22 06:02:31 2008 From: kuku at kukulies.org (Christoph Kukulies) Date: Sat Nov 22 06:02:40 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <863ahjc16w.fsf@ds4.des.no> References: <4927E659.3010602@kukulies.org> <863ahjc16w.fsf@ds4.des.no> Message-ID: <492810C8.6010109@kukulies.org> Dag-Erling Sm?rgrav schrieb: > Christoph Kukulies writes: > >> Since the underlying OS of Mac OS is FreeBSD >> > > It isn't, as someone else pointed out, and hence... > > Always thought that at some point in time it was derived from an earlier version of FreeBSD (4.x) >> I'm wondering whether itr is possible to mount a .dmg file as it is >> used in Apple software distribution for the Mac OS. I would like to >> install an Apple iPhone configuration utility on my FreeBSD box >> (iPhoneConfigurationUtility.dmg). >> > > ...Mac OS applications won't run on FreeBSD, so there's no point in even > trying. > It may be some java stuff. I'm not sure though since I didn't peek into the dmg-file yet. > DES > Thanks anyway, Christoph From ertr1013 at student.uu.se Sat Nov 22 06:36:25 2008 From: ertr1013 at student.uu.se (Erik Trulsson) Date: Sat Nov 22 06:36:32 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <492810C8.6010109@kukulies.org> References: <4927E659.3010602@kukulies.org> <863ahjc16w.fsf@ds4.des.no> <492810C8.6010109@kukulies.org> Message-ID: <20081122143620.GA88245@owl.midgard.homeip.net> On Sat, Nov 22, 2008 at 03:01:44PM +0100, Christoph Kukulies wrote: > Dag-Erling Sm?rgrav schrieb: > > Christoph Kukulies writes: > > > >> Since the underlying OS of Mac OS is FreeBSD > >> > > > > It isn't, as someone else pointed out, and hence... > > > > > Always thought that at some point in time it was derived from an earlier > version of FreeBSD (4.x) Parts of it was derived from FreeBSD (mainly userland stuff.) Other parts of MacOS X (including the kernel) was mainly derived from Mach, and some parts were of course written by Apple themselves (or taken from other places.) So, even though it is in part derived from FreeBSD, you should not expect any sort of binary compatibility between them. -- Erik Trulsson ertr1013@student.uu.se From ap at bnc.net Sat Nov 22 07:58:59 2008 From: ap at bnc.net (Achim Patzner) Date: Sat Nov 22 07:59:07 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <492810C8.6010109@kukulies.org> References: <4927E659.3010602@kukulies.org> <863ahjc16w.fsf@ds4.des.no> <492810C8.6010109@kukulies.org> Message-ID: <89C78BA8-5133-4B84-B893-D7826233B54B@bnc.net> Am 22.11.2008 um 15:01 schrieb Christoph Kukulies: >>> I would like to >>> install an Apple iPhone configuration utility on my FreeBSD box >>> (iPhoneConfigurationUtility.dmg). >> ...Mac OS applications won't run on FreeBSD, so there's no point in >> even >> trying. > It may be some java stuff. It is not. It's a Cocoa application. > I'm not sure I am. Achim From des at des.no Sat Nov 22 08:05:49 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Sat Nov 22 08:05:57 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <20081122143620.GA88245@owl.midgard.homeip.net> (Erik Trulsson's message of "Sat, 22 Nov 2008 15:36:20 +0100") References: <4927E659.3010602@kukulies.org> <863ahjc16w.fsf@ds4.des.no> <492810C8.6010109@kukulies.org> <20081122143620.GA88245@owl.midgard.homeip.net> Message-ID: <86y6zbagpw.fsf@ds4.des.no> Erik Trulsson writes: > Parts of it was derived from FreeBSD (mainly userland stuff.) Other > parts of MacOS X (including the kernel) was mainly derived from Mach, > and some parts were of course written by Apple themselves (or taken > from other places.) You got it almost entirely wrong... There are significant amounts of FreeBSD code in the Mac OS X kernel (the network stack, the VFS layer, several file systems, access control, security auditing and more). Mach provides only IPC, scheduling and virtual memory, IIRC. The rest of the kernel (including all device drivers) is proprietary Apple code. DES -- Dag-Erling Sm?rgrav - des@des.no From yanefbsd at gmail.com Sat Nov 22 14:06:41 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Sat Nov 22 14:06:48 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <4927E659.3010602@kukulies.org> References: <4927E659.3010602@kukulies.org> Message-ID: <7d6fde3d0811221406u38de767fnd22175518b8502bf@mail.gmail.com> On Sat, Nov 22, 2008 at 3:00 AM, Christoph Kukulies wrote: > Since the underlying OS of Mac OS is FreeBSD I'm wondering whether itr is > possible to mount a > .dmg file as it is used in Apple software distribution for the Mac OS. I > would like to install an > Apple iPhone configuration utility on my FreeBSD box > (iPhoneConfigurationUtility.dmg). It consists of a > web application that is supposedly capable of doing updates of Apple iPhones > through a web service. > (good for company wide distribution of address book information and other > data that iPhones can hold). > > -- > Christoph Kukulies As everyone else has pointed out, FreeBSD != OSX. I think that was the whole point of the Darwin logo (a platypus with Beastie's horns on it: ). If you watch the "BSD is Dying" presentation, you'll laugh when you see the reference to OSX as well: http://video.google.com/videoplay?docid=7833143728685685343 :). You have two choices: 1. Buy an Xserve loaded with OSX: http://shop.apple.com/ . 2. Compile and maintain your own copy of Apple's version of Darwin: http://opensource.apple.com/ . FYI, according to the Apple folks OSX is based off FreeBSD 5 (not sure which minor version): [Quote -- http://opensource.apple.com]: This fully-conformant UNIX operating system?built on Mach 3.0 and FreeBSD 5?bundles over a hundred of the most popular Open Source products. [/Quote] Cheers, -Garrett From ertr1013 at student.uu.se Sat Nov 22 14:23:34 2008 From: ertr1013 at student.uu.se (Erik Trulsson) Date: Sat Nov 22 14:23:40 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <7d6fde3d0811221406u38de767fnd22175518b8502bf@mail.gmail.com> References: <4927E659.3010602@kukulies.org> <7d6fde3d0811221406u38de767fnd22175518b8502bf@mail.gmail.com> Message-ID: <20081122222324.GA92500@owl.midgard.homeip.net> On Sat, Nov 22, 2008 at 02:06:39PM -0800, Garrett Cooper wrote: > On Sat, Nov 22, 2008 at 3:00 AM, Christoph Kukulies wrote: > > Since the underlying OS of Mac OS is FreeBSD I'm wondering whether itr is > > possible to mount a > > .dmg file as it is used in Apple software distribution for the Mac OS. I > > would like to install an > > Apple iPhone configuration utility on my FreeBSD box > > (iPhoneConfigurationUtility.dmg). It consists of a > > web application that is supposedly capable of doing updates of Apple iPhones > > through a web service. > > (good for company wide distribution of address book information and other > > data that iPhones can hold). > > > > -- > > Christoph Kukulies > > As everyone else has pointed out, FreeBSD != OSX. I think that was the > whole point of the Darwin logo (a platypus with Beastie's horns on it: > ). If you watch the > "BSD is Dying" presentation, you'll laugh when you see the reference > to OSX as well: > http://video.google.com/videoplay?docid=7833143728685685343 :). > > You have two choices: > 1. Buy an Xserve loaded with OSX: http://shop.apple.com/ . > 2. Compile and maintain your own copy of Apple's version of Darwin: > http://opensource.apple.com/ . > > FYI, according to the Apple folks OSX is based off FreeBSD 5 (not sure > which minor version): > > [Quote -- http://opensource.apple.com]: > This fully-conformant UNIX operating system?built on Mach 3.0 and > FreeBSD 5?bundles over a hundred of the most popular Open Source > products. > [/Quote] Which version of FreeBSD was used in OSX will almost certainly depend on which version of MacOS X you look at. It is quite possible that the latest version of OSX uses code from FreeBSD 5.x, but I guarantee that the first release of OSX did not. (This is easily seen from the fact the MacOS X 10.0 was first released in March 2001. The latest FreeBSD release at that time was 4.2 (released in November 2000.) FreeBSD 5.0 was not released until January 2003.) -- Erik Trulsson ertr1013@student.uu.se From rb at gid.co.uk Sun Nov 23 03:39:27 2008 From: rb at gid.co.uk (Bob Bishop) Date: Sun Nov 23 03:39:39 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <4927E659.3010602@kukulies.org> References: <4927E659.3010602@kukulies.org> Message-ID: Hi, On 22 Nov 2008, at 11:00, Christoph Kukulies wrote: > Since the underlying OS of Mac OS is FreeBSD I'm wondering whether > itr is possible to mount a > .dmg file as it is used in Apple software distribution for the Mac > OS. I would like to install an > Apple iPhone configuration utility on my FreeBSD box > (iPhoneConfigurationUtility.dmg). It consists of a > web application that is supposedly capable of doing updates of Apple > iPhones through a web service. > (good for company wide distribution of address book information and > other data that iPhones can hold). If you want to see what the package contains, try building dmg2img from here: http://vu1tur.eu.org/tools/ Use that to extract the HFS image from the package, then examine it one of the HFS tools from ports/emulators, But as others have mentioned, you may not be able to do much with the contents. -- Bob Bishop rb@gid.co.uk From rwatson at FreeBSD.org Sun Nov 23 07:23:34 2008 From: rwatson at FreeBSD.org (Robert Watson) Date: Sun Nov 23 07:23:40 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <20081122222324.GA92500@owl.midgard.homeip.net> References: <4927E659.3010602@kukulies.org> <7d6fde3d0811221406u38de767fnd22175518b8502bf@mail.gmail.com> <20081122222324.GA92500@owl.midgard.homeip.net> Message-ID: On Sat, 22 Nov 2008, Erik Trulsson wrote: > Which version of FreeBSD was used in OSX will almost certainly depend on > which version of MacOS X you look at. > > It is quite possible that the latest version of OSX uses code from FreeBSD > 5.x, but I guarantee that the first release of OSX did not. (This is easily > seen from the fact the MacOS X 10.0 was first released in March 2001. The > latest FreeBSD release at that time was 4.2 (released in November 2000.) > FreeBSD 5.0 was not released until January 2003.) It would be accurate to describe Apple's approach to adopting FreeBSD changes as "cherry-picking" - their source code base is sufficiently diverged from ours that they basicaly pull our change on an as-needed basis rather than remaining strictly sync'd to a particular version. Many of you will have noticed that articles describing the recent SnowLeopard seed include mention of pulling the most recent OpenBSM and OpenPAM drops from FreeBSD into Mac OS X, as examples of recently integrated technologies from recent FreeBSD versions. Robert N M Watson Computer Laboratory University of Cambridge From braulio at solsoft.co.cr Sun Nov 23 17:46:10 2008 From: braulio at solsoft.co.cr (Braulio =?iso-8859-1?Q?Jos=E9_Solano_Rojas?=) Date: Sun Nov 23 17:46:17 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: References: <4927E659.3010602@kukulies.org> Message-ID: <44979.201.198.34.106.1227489354.squirrel@webmail7.pair.com> On Sun, 23 de Noviembre de 2008, 5:22 am, Bob Bishop wrote: > > On 22 Nov 2008, at 11:00, Christoph Kukulies wrote: >> Since the underlying OS of Mac OS is FreeBSD I'm wondering whether >> itr is possible to mount a >> .dmg file as it is used in Apple software distribution for the Mac >> OS. I would like to install an >> Apple iPhone configuration utility on my FreeBSD box >> (iPhoneConfigurationUtility.dmg). It consists of a >> web application that is supposedly capable of doing updates of Apple >> iPhones through a web service. >> (good for company wide distribution of address book information and >> other data that iPhones can hold). > > ... > > But as others have mentioned, you may not be able to do much with the > contents. I have an innocent question. I have read on the handbook and the thesis about the Linux ABI technical explanations that lead me to think that it could be possible to run Mac OS binaries on FreeBSD. I think that if the correct loader was implemented in execve and the Mac OS system calls were also implemented it would be possible to run Mac OS binaries. Am I right? (I am not asking anyone to support this feature, I just would like an hypothetic answer in order to improve my knowledge.) Of course, I suppose there would be technical challenges to solve as there are still for Linux. In fact, I would like to ask further: could this be possible for any operating system (without thinking about if it would worth it)? Best regards, Braulio Solano From ken at mthelicon.com Sun Nov 23 17:52:16 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Sun Nov 23 17:52:22 2008 Subject: who on FreeBSD 8.0 - AMD64 Message-ID: <200811240152.13032.ken@mthelicon.com> Hello Hackers, I noticed something the other day that I thought was a bit weird. I had a high amount of HD activity, so I wanted to see who was on my machine. I tried using the "who" command and received no information (not even for myself). I have found that various methods of signing in cause different behavior, presumably due to being assigned different terminal names. For instance: If I open an Xterm, or telnet into the machine, I will not see any information (Although a who -q does), however, if I SSH or sign in on the console, I get the results I was expecting. Am I missing something or being a bit thick? (please see below for examples) Peg [through a xterm] feathers$ who feathers$ [through a telnet] feathers$ telnet localhost Trying ::1... Connected to localhost. Escape character is '^]'. Trying SRA secure login: User (XXXXXX): Password: [ SRA accepts you ] FreeBSD/amd64 (feathers.peganest.com) (pts/2) You have new mail. feathers$ who feathers$ [Through a SSH connection] titan$ ssh feathers Enter passphrase for key '/home/XXXXX/.ssh/id_rsa': Last login: Mon Nov 24 01:45:08 2008 from localhost Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 8.0-CURRENT (FEATHERS) #1: Sat Nov 22 16:07:04 UTC 2008 feathers$ feathers$ who XXXXX pts/3 Nov 24 01:47 (XXX.XXX.XXX.XXX) feathers$ From yanefbsd at gmail.com Sun Nov 23 18:16:09 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Sun Nov 23 18:16:16 2008 Subject: who on FreeBSD 8.0 - AMD64 In-Reply-To: <200811240152.13032.ken@mthelicon.com> References: <200811240152.13032.ken@mthelicon.com> Message-ID: <2B28B3C6-FC8F-4F00-A5CB-241DE2B8B00A@gmail.com> On Nov 23, 2008, at 5:52 PM, Pegasus Mc Cleaft wrote: > Hello Hackers, > > I noticed something the other day that I thought was a bit weird. I > had a > high amount of HD activity, so I wanted to see who was on my > machine. I tried > using the "who" command and received no information (not even for > myself). > > I have found that various methods of signing in cause different > behavior, > presumably due to being assigned different terminal names. For > instance: If I > open an Xterm, or telnet into the machine, I will not see any > information > (Although a who -q does), however, if I SSH or sign in on the > console, I get > the results I was expecting. > > Am I missing something or being a bit thick? (please see below for > examples) > > Peg > > [through a xterm] > feathers$ who > feathers$ > > [through a telnet] > feathers$ telnet localhost > Trying ::1... > Connected to localhost. > Escape character is '^]'. > Trying SRA secure login: > User (XXXXXX): > Password: > [ SRA accepts you ] > > FreeBSD/amd64 (feathers.peganest.com) (pts/2) > > You have new mail. > feathers$ who > feathers$ > > [Through a SSH connection] > titan$ ssh feathers > Enter passphrase for key '/home/XXXXX/.ssh/id_rsa': > Last login: Mon Nov 24 01:45:08 2008 from localhost > Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 > The Regents of the University of California. All rights > reserved. > > FreeBSD 8.0-CURRENT (FEATHERS) #1: Sat Nov 22 16:07:04 UTC 2008 > > > feathers$ > feathers$ who > XXXXX pts/3 Nov 24 01:47 (XXX.XXX.XXX.XXX) > feathers$ `who' works perfectly fine for me on CURRENT/AMD64 synced up 3 days ago, via an ssh terminal. Can you provide more details, e.g. when was your last sync date, did you blow away your utmp directory, etc? Thanks, -Garrett From ken at mthelicon.com Sun Nov 23 18:27:00 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Sun Nov 23 18:27:07 2008 Subject: who on FreeBSD 8.0 - AMD64 In-Reply-To: <2B28B3C6-FC8F-4F00-A5CB-241DE2B8B00A@gmail.com> References: <200811240152.13032.ken@mthelicon.com> <2B28B3C6-FC8F-4F00-A5CB-241DE2B8B00A@gmail.com> Message-ID: <200811240226.58137.ken@mthelicon.com> On Monday 24 November 2008 02:19:40 Garrett Cooper wrote: > On Nov 23, 2008, at 5:52 PM, Pegasus Mc Cleaft wrote: > > Hello Hackers, > > > > `who' works perfectly fine for me on CURRENT/AMD64 synced up 3 days > ago, via an ssh terminal. Can you provide more details, e.g. when was > your last sync date, did you blow away your utmp directory, etc? > Thanks, > -Garrett Hi Garrett, Its actually the other way around. If I SSH into the machine, I see my login in the 'who', however, if I telnet in or use a xterm I dont.. The last sync/build world/build kernel I have done was about 24 hours ago, however, I think the problem has been going on for a while now. I noticed it about a month ago and forgot about it until the other day when I tried using 'who' again. I havent blown away the utmp directory (or touched it, for that matter). I can try clearing if, if you think it may help? Ta Peg From lg.zevlg at gmail.com Sun Nov 23 15:45:27 2008 From: lg.zevlg at gmail.com (Zajcev Evgeny) Date: Sun Nov 23 18:37:25 2008 Subject: `type' built-in command in /bin/sh Message-ID: <87vdueavft.fsf@h1.lan> Hello there. I just hit into "problem" that built-in sh command type outputs to stdout when error occurs. I mean it returns non-0 status, but outputs to stdout! I think stderr has been made exactly for such purposes and `type' should use it in case of error .. what do you think? thanks -- lg From yanefbsd at gmail.com Sun Nov 23 18:42:08 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Sun Nov 23 18:42:15 2008 Subject: who on FreeBSD 8.0 - AMD64 In-Reply-To: <200811240226.58137.ken@mthelicon.com> References: <200811240152.13032.ken@mthelicon.com> <2B28B3C6-FC8F-4F00-A5CB-241DE2B8B00A@gmail.com> <200811240226.58137.ken@mthelicon.com> Message-ID: <7d6fde3d0811231842u418ac567m5cb2041b2cc8f1de@mail.gmail.com> On Sun, Nov 23, 2008 at 6:26 PM, Pegasus Mc Cleaft wrote: > On Monday 24 November 2008 02:19:40 Garrett Cooper wrote: >> On Nov 23, 2008, at 5:52 PM, Pegasus Mc Cleaft wrote: >> > Hello Hackers, >> > > > >> >> `who' works perfectly fine for me on CURRENT/AMD64 synced up 3 days >> ago, via an ssh terminal. Can you provide more details, e.g. when was >> your last sync date, did you blow away your utmp directory, etc? >> Thanks, >> -Garrett > > Hi Garrett, > Its actually the other way around. If I SSH into the machine, I see my login > in the 'who', however, if I telnet in or use a xterm I dont.. > > The last sync/build world/build kernel I have done was about 24 hours ago, > however, I think the problem has been going on for a while now. I noticed it > about a month ago and forgot about it until the other day when I tried using > 'who' again. > > I havent blown away the utmp directory (or touched it, for that matter). I > can try clearing if, if you think it may help? > > Ta > Peg Actually, no please don't blow it. After hearing that it occurs only over ssh I tried with a local login and it functions exactly the same as with the ssh terminal. 1. What's the exit code after you try and execute who? 2. `ls -l /var/run/utmp'? Should be set to mode 00644. Thanks, -Garrett From ken at mthelicon.com Sun Nov 23 18:49:21 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Sun Nov 23 18:49:28 2008 Subject: who on FreeBSD 8.0 - AMD64 In-Reply-To: <7d6fde3d0811231842u418ac567m5cb2041b2cc8f1de@mail.gmail.com> References: <200811240152.13032.ken@mthelicon.com> <200811240226.58137.ken@mthelicon.com> <7d6fde3d0811231842u418ac567m5cb2041b2cc8f1de@mail.gmail.com> Message-ID: <200811240249.18860.ken@mthelicon.com> On Monday 24 November 2008 02:42:08 you wrote: > On Sun, Nov 23, 2008 at 6:26 PM, Pegasus Mc Cleaft wrote: > > On Monday 24 November 2008 02:19:40 Garrett Cooper wrote: > >> On Nov 23, 2008, at 5:52 PM, Pegasus Mc Cleaft wrote: > >> > Hello Hackers, > > > > > > > >> `who' works perfectly fine for me on CURRENT/AMD64 synced up 3 > >> days ago, via an ssh terminal. Can you provide more details, e.g. when > >> was your last sync date, did you blow away your utmp directory, etc? > >> Thanks, > >> -Garrett > > > > Hi Garrett, > > Its actually the other way around. If I SSH into the machine, I > > see my login in the 'who', however, if I telnet in or use a xterm I > > dont.. > > > > The last sync/build world/build kernel I have done was about 24 > > hours ago, however, I think the problem has been going on for a while > > now. I noticed it about a month ago and forgot about it until the other > > day when I tried using 'who' again. > > > > I havent blown away the utmp directory (or touched it, for that > > matter). I can try clearing if, if you think it may help? > > > > Ta > > Peg > > Actually, no please don't blow it. After hearing that it occurs > only over ssh I tried with a local login and it functions exactly the > same as with the ssh terminal. > 1. What's the exit code after you try and execute who? > 2. `ls -l /var/run/utmp'? Should be set to mode 00644. > Thanks, > -Garrett Hi Garrett, I'm not sure of the exit code, but it dosent display any errors, just returns nothing. The /var/run/utmp is set to 00644 as well.. Can you try telneting into your box (even as a localhost) and see if if it still functions correctly? Using a telnet login or a xterm window is the only time I see it fail. Ta Peg From mwm-keyword-freebsdhackers2.e313df at mired.org Sun Nov 23 21:48:26 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Sun Nov 23 21:48:32 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <44979.201.198.34.106.1227489354.squirrel@webmail7.pair.com> References: <4927E659.3010602@kukulies.org> <44979.201.198.34.106.1227489354.squirrel@webmail7.pair.com> Message-ID: <20081124002133.500eec9e@bhuda.mired.org> On Sun, 23 Nov 2008 19:15:54 -0600 (CST) Braulio Jos? Solano Rojas wrote: > I have an innocent question. I have read on the handbook and the thesis > about the Linux ABI technical explanations that lead me to think that it > could be possible to run Mac OS binaries on FreeBSD. I think that if the > correct loader was implemented in execve and the Mac OS system calls were > also implemented it would be possible to run Mac OS binaries. Am I right? > (I am not asking anyone to support this feature, I just would like an > hypothetic answer in order to improve my knowledge.) Of course, I suppose > there would be technical challenges to solve as there are still for Linux. > In fact, I would like to ask further: could this be possible for any > operating system (without thinking about if it would worth it)? In theory, you can provide an ABI for any OS that you care to emulate the necessary calls for. In essence, that's what the WINE project is: emulation of all the Windows API's required to run Windows applications on a Unix/X11 platform. The thing is, an OS these days is more than just a set of system calls - it's a collection of shared libraries providing a complete user interface. That's why you have to install large chunks of a Linux system to run Linux binaries on FreeBSD - getting all the libraries down to the system call level needed by those binaries (and in some cases, you want GNU/Linux executable also, because the binaries expect to invoke executables from Linux, not BSD, and they are different enough to matter). Even then, the reason it works reasonably well for GNU/Linux with X is because the X server/client API use IPC mechanisms so you can get away with running a FreeBSD X11 server, letting the Linux applications use Linux client-side libraries, and just make sure the IPC calls are simulated properly (module hi-end graphics tools and other late additions). Other OSs don't have as clean a division between applications and the graphics subsystem, so you have to provide all the services the OS provides for talking to the graphics device as well. Linux gets another win here in that all the relevant libraries are open source; for proprietary systems, even if you could find a clean layer to switch platforms, you'd have to rewrite all those libraries above that layer from scratch anyway. Which is why WINE has to simulate all the Windows GUI calls using X11 code. So while a Darwin (the OS underneath OSX) ABI would be possible - though it's not clear how painful because it's not clear how visible the MACH APIs are - it's not clear how useful it would be by itself. You could run OSX applications that used X11 for the GUI after building the appropriate libraries (just like Linux), but those apps are probably available native anyway. You couldn't run applications that use the various and sundry Mac-specific graphics (among other things) frameworks, not without providing code to simulate all the calls provided by those frameworks - which are proprietary, and not part of Darwin. Which means this project now resembles WINE more than the Linux ABI layer. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From leafy7382 at gmail.com Sun Nov 23 22:40:49 2008 From: leafy7382 at gmail.com (Jiawei Ye) Date: Sun Nov 23 22:40:56 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: <20081124002133.500eec9e@bhuda.mired.org> References: <4927E659.3010602@kukulies.org> <44979.201.198.34.106.1227489354.squirrel@webmail7.pair.com> <20081124002133.500eec9e@bhuda.mired.org> Message-ID: On Mon, Nov 24, 2008 at 1:21 PM, Mike Meyer wrote: --snip-- > So while a Darwin (the OS underneath OSX) ABI would be possible - > though it's not clear how painful because it's not clear how visible > the MACH APIs are - it's not clear how useful it would be by > itself. You could run OSX applications that used X11 for the GUI after > building the appropriate libraries (just like Linux), but those apps > are probably available native anyway. You couldn't run applications > that use the various and sundry Mac-specific graphics (among other > things) frameworks, not without providing code to simulate all the > calls provided by those frameworks - which are proprietary, and not > part of Darwin. Which means this project now resembles WINE more than > the Linux ABI layer. > > References: <87vdueavft.fsf@h1.lan> Message-ID: <20081124084016.GA1384@lizard.fafoe.narf.at> On Mon, Nov 24, 2008 at 02:12:22AM +0300, Zajcev Evgeny wrote: > > Hello there. I just hit into "problem" that built-in sh command type > outputs to stdout when error occurs. I mean it returns non-0 status, > but outputs to stdout! I think stderr has been made exactly for such > purposes and `type' should use it in case of error .. what do you > think? You're right. I will fix it. From dfr at rabson.org Mon Nov 24 02:35:21 2008 From: dfr at rabson.org (Doug Rabson) Date: Mon Nov 24 02:35:28 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <367b2c980811201141j42977204ne6052000a0d095ab@mail.gmail.com> References: <20081120134836.2870a827@nebuchadnezzar> <367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com> <367b2c980811201107x7fc859b8yeee0816a37eae470@mail.gmail.com> <367b2c980811201141j42977204ne6052000a0d095ab@mail.gmail.com> Message-ID: <16BCBB6C-2BF4-4007-9A19-9976BEBAAD1E@rabson.org> On 20 Nov 2008, at 19:41, Olivier SMEDTS wrote: > 2008/11/20 Olivier SMEDTS : >> 2008/11/20 Olivier SMEDTS : >>> 2008/11/20 Pascal Hofstee : >>>> On Thu, 20 Nov 2008 01:46:31 -0000 >>>> "Pegasus Mc Cleaft" wrote: >>>> >>>>> Hi everyone, >>>>> >>>>> I am having difficulties rebuilding the world after some >>>>> patches >>>>> were made today. I was wondering if anyone else is experiencing >>>>> the >>>>> same troubles? >>>>> >>>>> ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext >>>>> 0x0 -o >>>>> gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/ >>>>> lib/crt0.o >>>>> zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No >>>>> such >>>>> file or directory *** Error code 1 >>>>> >>>>> Stop in /usr/src/sys/boot/i386/gptzfsboot. >>>>> *** Error code 1 >>>> >>>> I am experiencing the exact same problem with a fresh svn checkout >>> >>> Just my "me too". >>> I did not experience the problem 24 hours ago (after ZFS version 13 >>> update and zfsboot import). >> >> That's it. Seems to work with the following patch : >> >> --- sys/boot/i386/gptzfsboot/Makefile.orig 2008-11-20 >> 19:58:45.000000000 +0100 >> +++ sys/boot/i386/gptzfsboot/Makefile 2008-11-20 >> 20:01:53.000000000 +0100 >> @@ -65,7 +65,7 @@ >> zfsboot.o: ${.CURDIR}/../../zfs/zfsimpl.c >> >> .if ${MACHINE_ARCH} == "amd64" >> -beforedepend gptzfsboot.o: machine >> +beforedepend gptzfsboot.bin: machine >> CLEANFILES+= machine >> machine: >> ln -sf ${.CURDIR}/../../../i386/include machine > > Sorry for replying again to my own post :) > The patch is crap, in fact it just breaks the already broken > conditional. At least I can buildworld on amd64 now (I don't use the > recently introduced gptzfsboot). > > Makefile experts ? I think I fixed this in r185175. Sorry for the inconvenience. From ken at mthelicon.com Mon Nov 24 02:41:24 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Mon Nov 24 02:41:31 2008 Subject: build problems with gptzfsboot (AMD64) 8.0-CURRENT In-Reply-To: <16BCBB6C-2BF4-4007-9A19-9976BEBAAD1E@rabson.org> References: <20081120134836.2870a827@nebuchadnezzar><367b2c980811201038s7d2ae03bnf36a6630f36bc188@mail.gmail.com><367b2c980811201107x7fc859b8yeee0816a37eae470@mail.gmail.com><367b2c980811201141j42977204ne6052000a0d095ab@mail.gmail.com> <16BCBB6C-2BF4-4007-9A19-9976BEBAAD1E@rabson.org> Message-ID: <0B3A85C390F2454C95A3BE66FDC95DAE@PegaPegII> ----- Original Message ----- From: "Doug Rabson" To: "Olivier SMEDTS" Cc: ; ; "Pegasus Mc Cleaft" ; "Pascal Hofstee" Sent: Monday, November 24, 2008 10:35 AM Subject: Re: build problems with gptzfsboot (AMD64) 8.0-CURRENT > > On 20 Nov 2008, at 19:41, Olivier SMEDTS wrote: > >> 2008/11/20 Olivier SMEDTS : >>> 2008/11/20 Olivier SMEDTS : >>>> 2008/11/20 Pascal Hofstee : >>>>> On Thu, 20 Nov 2008 01:46:31 -0000 >>>>> "Pegasus Mc Cleaft" wrote: >>>>> >>>>>> Hi everyone, >>>>>> >>>>>> I am having difficulties rebuilding the world after some patches >>>>>> were made today. I was wondering if anyone else is experiencing the >>>>>> same troubles? >>>>>> >>>>>> ld -static -N --gc-sections -nostdlib -m elf_i386_fbsd -Ttext 0x0 -o >>>>>> gptzfsboot.out /usr/obj/usr/src/sys/boot/i386/gptzfsboot/../btx/ >>>>>> lib/crt0.o >>>>>> zfsboot.o sio.o gptzfsboot.o ld: gptzfsboot.o: No such file: No such >>>>>> file or directory *** Error code 1 >>>>>> >>>>>> Stop in /usr/src/sys/boot/i386/gptzfsboot. >>>>>> *** Error code 1 >>>>> >>>>> I am experiencing the exact same problem with a fresh svn checkout >>>> >>>> Just my "me too". >>>> I did not experience the problem 24 hours ago (after ZFS version 13 >>>> update and zfsboot import). >>> >>> That's it. Seems to work with the following patch : >>> >>> --- sys/boot/i386/gptzfsboot/Makefile.orig 2008-11-20 >>> 19:58:45.000000000 +0100 >>> +++ sys/boot/i386/gptzfsboot/Makefile 2008-11-20 20:01:53.000000000 >>> +0100 >>> @@ -65,7 +65,7 @@ >>> zfsboot.o: ${.CURDIR}/../../zfs/zfsimpl.c >>> >>> .if ${MACHINE_ARCH} == "amd64" >>> -beforedepend gptzfsboot.o: machine >>> +beforedepend gptzfsboot.bin: machine >>> CLEANFILES+= machine >>> machine: >>> ln -sf ${.CURDIR}/../../../i386/include machine >> >> Sorry for replying again to my own post :) >> The patch is crap, in fact it just breaks the already broken >> conditional. At least I can buildworld on amd64 now (I don't use the >> recently introduced gptzfsboot). >> >> Makefile experts ? > > I think I fixed this in r185175. Sorry for the inconvenience. > Hi Doug, Yes and thank you. I am able to build the world now :> No appologies necessary, I'm just glad it wasent something I did on my machine (The build failure happened exactly at the same time I moved /usr/src and /usr/obj into a seperate zfs.. -Peg From vadim_nuclight at mail.ru Mon Nov 24 04:20:10 2008 From: vadim_nuclight at mail.ru (Vadim Goncharov) Date: Mon Nov 24 04:20:18 2008 Subject: who on FreeBSD 8.0 - AMD64 References: <200811240152.13032.ken@mthelicon.com> Message-ID: Hi Pegasus Mc Cleaft! On Mon, 24 Nov 2008 01:52:12 +0000; Pegasus Mc Cleaft wrote about 'who on FreeBSD 8.0 - AMD64': > FreeBSD/amd64 (feathers.peganest.com) (pts/2) > > feathers$ who > XXXXX pts/3 Nov 24 01:47 (XXX.XXX.XXX.XXX) > feathers$ By the way, are there pseudo-terminal names renamed in -CURRENT ? Why? It is more cute to see ttyp0 or simply p0 on FreeBSD, compared to pts/1 on Linux :) -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] From rink at FreeBSD.org Mon Nov 24 04:45:19 2008 From: rink at FreeBSD.org (Rink Springer) Date: Mon Nov 24 04:45:26 2008 Subject: who on FreeBSD 8.0 - AMD64 In-Reply-To: References: <200811240152.13032.ken@mthelicon.com> Message-ID: <20081124122718.GB28709@rink.nu> On Mon, Nov 24, 2008 at 12:19:54PM +0000, Vadim Goncharov wrote: > By the way, are there pseudo-terminal names renamed in -CURRENT ? Why? Yes, this is the result of the new giant-free TTY layer that was imported a few months ago. Regards, -- Rink P.W. Springer - http://rink.nu "Anyway boys, this is America. Just because you get more votes doesn't mean you win." - Fox Mulder From braulio at solsoft.co.cr Mon Nov 24 07:44:16 2008 From: braulio at solsoft.co.cr (Braulio =?iso-8859-1?Q?Jos=E9_Solano_Rojas?=) Date: Mon Nov 24 07:44:23 2008 Subject: mounting Mac OS .dmg files? In-Reply-To: References: <4927E659.3010602@kukulies.org> <44979.201.198.34.106.1227489354.squirrel@webmail7.pair.com> <20081124002133.500eec9e@bhuda.mired.org> Message-ID: <47774.201.198.34.106.1227541453.squirrel@webmail7.pair.com> Hi! On Mon, 24 de Noviembre de 2008, 12:19 am, Jiawei Ye wrote: > On Mon, Nov 24, 2008 at 1:21 PM, Mike Meyer > wrote: > --snip-- >> So while a Darwin (the OS underneath OSX) ABI would be possible - >> though it's not clear how painful because it's not clear how visible >> the MACH APIs are - it's not clear how useful it would be by >> itself. You could run OSX applications that used X11 for the GUI after >> building the appropriate libraries (just like Linux), but those apps >> are probably available native anyway. You couldn't run applications >> that use the various and sundry Mac-specific graphics (among other >> things) frameworks, not without providing code to simulate all the >> calls provided by those frameworks - which are proprietary, and not >> part of Darwin. Which means this project now resembles WINE more than >> the Linux ABI layer. >> >> One open-source project at least try to replicate the frameworks on > top of Windows: > > http://www.cocotron.org/ > > As Mike mentioned this is not a trivial port and requires tweaks to > the target application source as well. So even if one is diligent > enough to get cocotron running on FreeBSD/Linux, the app itself will > have to be modified to run there. Thank you for your answers. This list is always very educative. It seems like it could take years to program a Mac ABI layer. Best regards, B. From ravi.murty at gmail.com Mon Nov 24 08:23:21 2008 From: ravi.murty at gmail.com (Ravi Murty) Date: Mon Nov 24 08:23:28 2008 Subject: option pci Message-ID: <95b10a340811240759gf6eb4e9qd6fd06a92bc3bc63@mail.gmail.com> Hello All, I've been trying to turn off option PCI and several other options in the configuration file when I build my 8.0 kernel. For things like "option INET", I see a bunch of link errors and after looking at the head of the tree realized that the latest source code has some of the functions wrapped in ifdef INET and include the proper headers. For option PCI, nexus.c and some of the pci device functionalily call functions that are defined in msi.c which is optional while the files that call these functions are defined "standard". Is there an easy way to cleanly compile the kernel with option pci turned off? Thanks Ravi From bf2006a at yahoo.com Tue Nov 25 03:23:37 2008 From: bf2006a at yahoo.com (bf) Date: Tue Nov 25 04:30:42 2008 Subject: lzma compression/decompression in bsdtar/libarchive? Message-ID: <259774.68895.qm@web39105.mail.mud.yahoo.com> Tim: There is good news: Igor Pavlov, the primary author of the original LZMA SDK, has placed the latest version, available at: http://downloads.sourceforge.net/sevenzip/lzma461.tar.bz2 into the public domain. It's a mix of ANSI-C and C++ code, and so it would seem suitable for adoption into the FreeBSD source tree in some form that could be integrated with bsdtar/libarchive. What do you think? It would be *really* nice to have this, since in many ways it is better than bzip2, and many projects have started to distribute code in lzma-compressed tarballs. It could help us save disk space and network throughput, and help us with the current problems in shoehorning releases onto as few cds as possible, etc. Regards, b. --- On Thu, 3/6/08, bf wrote: > From: bf > Subject: Re: bsdtar/libarchive > To: "Tim Kientzle" > Date: Thursday, March 6, 2008, 1:52 PM > Yes, the LZMA implementations that I have seen are > LGPL, CPL, and/or GPL: > > http://www.7-zip.org/sdk.html > http://tukaani.org/lzma/ > > I will inquire if anyone knows of a BSD-licensed > alternative. I was hoping that portmgr would make > some minimal changes to encourage people to take > advantage of the LZMA port distfiles that are now > available, but it would be nice to have it available > in the base, too. > > Regards, > > bf > > > > --- Tim Kientzle wrote: > > > This would first require incorporating a > > BSD-licensed LZMA > > library into the base system. Last I checked, the > > LZMA > > code was all GPL, but that may have changed. (I > > seem to recall finding suitable decompression code > > at one point, but that's not sufficient for this > > type > > of change.) > > > > After that hurdle, incorporating LZMA support into > > libarchive is pretty straightforward. Once > > libarchive > > supports it, bsdtar gets it pretty much for free. > > > > Tim > > > > bf wrote: > > > Perhaps you have some comments or suggestions?: > > > > > > > > > > > > http://www.FreeBSD.org/cgi/query-pr.cgi?pr=ports/121413 > > > > > > > > > Regards, > > > bf > > > From laladelausanne at gmail.com Tue Nov 25 07:06:28 2008 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Tue Nov 25 07:06:34 2008 Subject: How to build kernel module spread on subdirectories? Message-ID: <711D7381-D852-4B6B-991A-84BA6DEFB679@gmail.com> Hi, I'm playing with the Click Modular router on my FreeBSD box. Out of curiosity, I decided to switch its GNU makefile to BSD style. I managed to do it, but I would like to polish it a bit more (and learn some things along). Old GNUmakefile relies heavily on OBJS and *_OBJS, as it fills these variables. I would like to switch to using SRCS. I managed to do it partially, for the files which are in the ${.CURDIR}. Click has a certain dir hierarchy of elements. First, it builds its elements, where each group of elements is in a different directory. Then it places all generated .o files in ${.CURDIR}. Finally, it links the whole shebang. Element building is done in another Makefile, which is generated by a script. (Main) GNUmakefile calls this script, then it includes that makefile via '-include'. The script traverses the dir hierarchy, and based on some variables, creates a Makefile which outlines like this: # subdir0 ELEMENT_OBJS__x= \ file0.o \ file1.o #subdir1 ELEMENT_OBJS__y= \ file2.o \ file3.o \ ... ELEMENT_OBJS= \ $(ELEMENT_OBJS__x) \ $(ELEMENT_OBJS__y) $(ELEMENT_OBJS__x): %.o: subdir0/%.cc $(call cxxcompile,-c $< -o $@,CXX) $(ELEMENT_OBJS__y): %.o: subdir1/%.cc $(call cxxcompile,-c $< -o $@,CXX) I'm interested in how can I transfer this into BSD-style makefile? I tried to move from OBJS into SRCS (main BSDmakefile now has: SRCS+=$ (ELEMENT_SRCS)), by using something like: # subdir0 ELEMENT_SRCS__x =\ subdir1/file0.cc \ subdir1/file1.cc ... But this fails during the linking phase, because the linker is called with subdir1/file0.o, instead of just file0.o. To make something clear, I didn't just rewrite the GNUmakefile to BSDmakefile, I also followed some of the logic used to build kernel modules. I'm including bsd.kmod.ko, list sources in SRCS, don't have any explicit rule to build .o out of .cc/.c. There is no all: target, as well. Thanks, Nikola From psteele at maxiscale.com Tue Nov 25 08:53:43 2008 From: psteele at maxiscale.com (Peter Steele) Date: Tue Nov 25 08:53:50 2008 Subject: Hot swapping SATA drives Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD5E@polaris.maxiscale.com> I've done some searches regarding FreeBSD's 7 support for the hot swapping of SATA drives and the general consensus appears that it *is* supported, but not necessarily with all drive models/brands. In our own testing, we've discovered that our Seagate 250GB drives cannot be hot swapped in our servers. The system appears to sense when they are removed but not when they are reinserted, and we've had numerous panics experimenting with them. We also have some Western Digital drives, and these fare much better. FreeBSD appears to recognize when these drives are removed and inserted. If we have a WD configured as part of a geom mirror, the geom driver automatically re-inserts a previously configured drive as soon as it is plugged in. It isn't even necessary to do an atacontrol attach/detach. However, even with the Western Digital drive, there are issues. In particular, if there are any mounted file systems on a drive when it is removed, attempting to unmount the file systems after it has been removed usually leads to a kernel panic, not necessarily immediately but shortly afterwards. I've tried the latest 7.0 patch level, p6, and the panics appear to have been fixed, but there are still problems. If a drive dies on us, we want to be able to close existing file handles and allow the new drive to take over. But what we've experienced is that even a "umount -f" will not umount a file system if the drive has been pulled. And as I type this, I have a system in the lab that is completely frozen after a drive pull test. No panic, no reboot, it's just hung up solid. Why does FreeBSD panic/freeze instead of simply issuing an I/O error, and why is there no way to force open file handles to close when a drive is pulled. The implication is that if a drive was to suddenly die on a live system, even if we have gmirror configured for HA, the system will likely panic or freeze and we'll have to reboot. We have software that detects when a drive disappears, but if the system is going to end up having to be rebooted, our detection code isn't going to do us much good. Is there any solution to this? Can a server be built around FreeBSD that supports hot swappable SATA drives? From kientzle at freebsd.org Tue Nov 25 09:05:30 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Tue Nov 25 09:05:36 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <259774.68895.qm@web39105.mail.mud.yahoo.com> References: <259774.68895.qm@web39105.mail.mud.yahoo.com> Message-ID: <492C3053.1030802@freebsd.org> Where is the announcement of this change? I haven't downloaded the code yet, but the sourceforge project pages all still say GPL. Tim bf wrote: > Tim: > > There is good news: Igor Pavlov, the primary author of the original LZMA > SDK, has placed the latest version, available at: > > http://downloads.sourceforge.net/sevenzip/lzma461.tar.bz2 > > into the public domain. It's a mix of ANSI-C and C++ code, and so it > would seem suitable for adoption into the FreeBSD source tree in some > form that could be integrated with bsdtar/libarchive. What do you > think? It would be *really* nice to have this, since in many ways > it is better than bzip2, and many projects have started to distribute > code in lzma-compressed tarballs. It could help us save disk space > and network throughput, and help us with the current problems in > shoehorning releases onto as few cds as possible, etc. > > Regards, > b. > > > > --- On Thu, 3/6/08, bf wrote: > > >>From: bf >>Subject: Re: bsdtar/libarchive >>To: "Tim Kientzle" >>Date: Thursday, March 6, 2008, 1:52 PM >>Yes, the LZMA implementations that I have seen are >>LGPL, CPL, and/or GPL: >> >>http://www.7-zip.org/sdk.html >>http://tukaani.org/lzma/ >> >>I will inquire if anyone knows of a BSD-licensed >>alternative. I was hoping that portmgr would make >>some minimal changes to encourage people to take >>advantage of the LZMA port distfiles that are now >>available, but it would be nice to have it available >>in the base, too. >> >>Regards, >> >>bf >> >> >> >>--- Tim Kientzle wrote: >> >> >>>This would first require incorporating a >>>BSD-licensed LZMA >>>library into the base system. Last I checked, the >>>LZMA >>>code was all GPL, but that may have changed. (I >>>seem to recall finding suitable decompression code >>>at one point, but that's not sufficient for this >>>type >>>of change.) >>> >>>After that hurdle, incorporating LZMA support into >>>libarchive is pretty straightforward. Once >>>libarchive >>>supports it, bsdtar gets it pretty much for free. >>> >>>Tim >>> >>>bf wrote: >>> >>>>Perhaps you have some comments or suggestions?: >>>> >>>> >>>> >>> >>http://www.FreeBSD.org/cgi/query-pr.cgi?pr=ports/121413 >> >>>> >>>>Regards, >>>>bf >>>> > > > > > > > From fjwcash at gmail.com Tue Nov 25 09:29:39 2008 From: fjwcash at gmail.com (Freddie Cash) Date: Tue Nov 25 09:29:46 2008 Subject: Hot swapping SATA drives In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD5E@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD5E@polaris.maxiscale.com> Message-ID: <200811250929.37929.fjwcash@gmail.com> On November 25, 2008 08:53 am Peter Steele wrote: > Is there any solution to this? Can a server be built around FreeBSD > that supports hot swappable SATA drives? Use a real hot-swappable drive plane, attached to a good SATA controller that handles hot-swap in hardware? :) Use ZFS, which seems to work better with drives being added/removed than ata(4)? :) Sorry, the few systems we have running FreeBSD either have single IDE drives, single SATA drives, or 12-24 SATA drives attached to a hardware hot-swappable drive-plane connected to 3Ware 9550/9650 RAID controllers. The single-drive systems obviously can't do swapping, and the rest work without issues. -- Freddie Cash fjwcash@gmail.com From bsd.quest at googlemail.com Tue Nov 25 09:38:20 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Tue Nov 25 09:38:26 2008 Subject: copy, copyin, copyout Message-ID: <20081125173750.GA4131@debian.samsung.router> Hello, could anyone please explain to me the difference between functions: copystr() and copyinstr() ? For i386 copyinstr is implemented in assembler. I can not read the assembler code very well. I tried to allocate a memory in space of user process using vm_map_find and then with copystr() I could copy data between user and kernel memory. copystr() seemed to be able to do the same what copyinstr do. % uname -rp 7.0-RELEASE-p5 i386 -- Alexej Sokolov From scf at FreeBSD.org Tue Nov 25 09:41:35 2008 From: scf at FreeBSD.org (Sean C. Farley) Date: Tue Nov 25 09:41:41 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <492C3053.1030802@freebsd.org> References: <259774.68895.qm@web39105.mail.mud.yahoo.com> <492C3053.1030802@freebsd.org> Message-ID: On Tue, 25 Nov 2008, Tim Kientzle wrote: > Where is the announcement of this change? > > I haven't downloaded the code yet, but the > sourceforge project pages all still say GPL. It is on the SDK page: http://www.7-zip.org/sdk.html > bf wrote: >> Tim: >> >> There is good news: Igor Pavlov, the primary author of the original >> LZMA SDK, has placed the latest version, available at: >> >> http://downloads.sourceforge.net/sevenzip/lzma461.tar.bz2 >> >> into the public domain. It's a mix of ANSI-C and C++ code, and so it >> would seem suitable for adoption into the FreeBSD source tree in some >> form that could be integrated with bsdtar/libarchive. What do you >> think? It would be *really* nice to have this, since in many ways it >> is better than bzip2, and many projects have started to distribute >> code in lzma-compressed tarballs. It could help us save disk space >> and network throughput, and help us with the current problems in >> shoehorning releases onto as few cds as possible, etc. *snip* Sean -- scf@FreeBSD.org From max at love2party.net Tue Nov 25 09:55:50 2008 From: max at love2party.net (Max Laier) Date: Tue Nov 25 09:55:58 2008 Subject: copy, copyin, copyout In-Reply-To: <20081125173750.GA4131@debian.samsung.router> References: <20081125173750.GA4131@debian.samsung.router> Message-ID: <200811251855.44853.max@love2party.net> On Tuesday 25 November 2008 18:37:50 Alexej Sokolov wrote: > Hello, > could anyone please explain to me the difference between functions: > copystr() and copyinstr() ? > > For i386 copyinstr is implemented in assembler. I can not read > the assembler code very well. > > I tried to allocate a memory in space of user process using vm_map_find > and then with copystr() I could copy data between user and kernel > memory. copystr() seemed to be able to do the same what copyinstr do. You might get lucky with copystr() if the user page is already resident, but if you page fault copystr() will kill the kernel. copyinstr() handles page faults. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From ivoras at freebsd.org Tue Nov 25 10:02:27 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Nov 25 10:02:33 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <259774.68895.qm@web39105.mail.mud.yahoo.com> References: <259774.68895.qm@web39105.mail.mud.yahoo.com> Message-ID: bf wrote: > Tim: > > There is good news: Igor Pavlov, the primary author of the original LZMA > SDK, has placed the latest version, available at: > > http://downloads.sourceforge.net/sevenzip/lzma461.tar.bz2 > > into the public domain. It's a mix of ANSI-C and C++ code, and so it > would seem suitable for adoption into the FreeBSD source tree in some > form that could be integrated with bsdtar/libarchive. What do you > think? It would be *really* nice to have this, since in many ways > it is better than bzip2, and many projects have started to distribute > code in lzma-compressed tarballs. It could help us save disk space > and network throughput, and help us with the current problems in > shoehorning releases onto as few cds as possible, etc. How useful would LZMA be without supporting the .7z file format? Probably not at all, since there isn't a gzip-like file format or wrapper that supports LZMA. Since .7z supports a lot of compression formats and options (unfortunatly saving unix ownership and mode attributes isn't one of them so be careful there!) it would probably be easier to just import p7z - and that's probably out of the question. It's available in the ports just fine. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 258 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081125/ecf8bdc0/signature.pgp From bsd.quest at googlemail.com Tue Nov 25 10:26:58 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Tue Nov 25 10:27:05 2008 Subject: copy, copyin, copyout In-Reply-To: <200811251855.44853.max@love2party.net> References: <20081125173750.GA4131@debian.samsung.router> <200811251855.44853.max@love2party.net> Message-ID: <20081125182627.GA4223@debian.samsung.router> On Tue, Nov 25, 2008 at 06:55:21PM +0100, Max Laier wrote: > On Tuesday 25 November 2008 18:37:50 Alexej Sokolov wrote: > > Hello, > > could anyone please explain to me the difference between functions: > > copystr() and copyinstr() ? > > > > For i386 copyinstr is implemented in assembler. I can not read > > the assembler code very well. > > > > I tried to allocate a memory in space of user process using vm_map_find > > and then with copystr() I could copy data between user and kernel > > memory. copystr() seemed to be able to do the same what copyinstr do. > > You might get lucky with copystr() if the user page is already resident, but > if you page fault copystr() will kill the kernel. copyinstr() handles page > faults. Thanks a lot! I was lucky because I wired allocated pages before. > > -- > /"\ Best regards, | mlaier@freebsd.org > \ / Max Laier | ICQ #67774661 > X http://pf4freebsd.love2party.net/ | mlaier@EFnet > / \ ASCII Ribbon Campaign | Against HTML Mail and News -- Alexej Sokolov From bf2006a at yahoo.com Tue Nov 25 10:56:25 2008 From: bf2006a at yahoo.com (bf) Date: Tue Nov 25 11:23:10 2008 Subject: lzma compression/decompression in bsdtar/libarchive? Message-ID: <917783.1650.qm@web39102.mail.mud.yahoo.com> > How useful would LZMA be without supporting the .7z file format? > Probably not at all, since there isn't a gzip-like file format or > wrapper that supports LZMA. ?? Have you looked at this code? Yes, there is: there is an "LZMA compressed file format" and the 7z file format, both of which support LZMA. The former format has been widely adopted by people who distribute lzma-compressed tarballs, especially GNU-related projects that use the lzmautils port. Some projects, like GNU coreutils, no longer distribute the latest versions of their software in bzip2-compressed tarballs. > Since .7z supports a lot of compression formats and options > (unfortunatly saving unix ownership and mode attributes isn't one of > them so be careful there!) it would probably be easier to just import > p7z - and that's probably out of the question. It's available in the > ports just fine. I don't see why the 7z file format couldn't be supported, if that's considered desirable. But it's certainly not *necessary* to do so in order to derive some benefit from lzma compression. It seems to me that importing some form of the smaller lzma sdk would be easier than importing the whole of p7z, which probably isn't necessary in any case. Both the lzma sdk and p7z are available in ports, but they haven't been exploited in ports or the base system. If there is an efficient, integrated lzma (de)compression engine in libarchive, it is almost certain to be more widely used, and that would probably benefit a lot of people. It's the same rationale for having compress, zip, gzip, and bzip2 forms of compression in the base system, although they are also available from ports. Regards, b. From psteele at maxiscale.com Tue Nov 25 11:42:08 2008 From: psteele at maxiscale.com (Peter Steele) Date: Tue Nov 25 11:42:16 2008 Subject: Hot swapping SATA drives In-Reply-To: <200811250929.37929.fjwcash@gmail.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD5E@polaris.maxiscale.com> <200811250929.37929.fjwcash@gmail.com> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD9D@polaris.maxiscale.com> >Use a real hot-swappable drive plane, attached to a good SATA controller >that handles hot-swap in hardware? :) > >Use ZFS, which seems to work better with drives being added/removed than >ata(4)? :) > >Sorry, the few systems we have running FreeBSD either have single IDE >drives, single SATA drives, or 12-24 SATA drives attached to a hardware >hot-swappable drive-plane connected to 3Ware 9550/9650 RAID controllers. >The single-drive systems obviously can't do swapping, and the rest work >without issues. I should further clarified that we are running 4-drive systems, with drive sizes ranging from 250GB-1TB. These drives are not in a RAID cluster and we do not want them to be. We do need the drives to be hot swappable though. I'll contact 3Ware and go from there. Thanks for the reply. From ivoras at freebsd.org Tue Nov 25 12:12:38 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Nov 25 12:12:44 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <917783.1650.qm@web39102.mail.mud.yahoo.com> References: <917783.1650.qm@web39102.mail.mud.yahoo.com> Message-ID: <9bbcef730811251141w63ad793as6efac3e7156bc2ef@mail.gmail.com> 2008/11/25 bf : >> How useful would LZMA be without supporting the .7z file format? >> Probably not at all, since there isn't a gzip-like file format or >> wrapper that supports LZMA. > > ?? Have you looked at this code? Yes, there is: there is an "LZMA > compressed file format" and the 7z file format, both of which support > LZMA. The former format has been widely adopted by people who distribute > lzma-compressed tarballs, especially GNU-related projects that use the > lzmautils port. Some projects, like GNU coreutils, no longer distribute > the latest versions of their software in bzip2-compressed tarballs. That's interesting - I've never seen an .lzma file "in the wild". But there they are: http://ftp.gnu.org/gnu/coreutils/ [ ] coreutils-6.12.tar.gz 01-Jun-2008 05:03 8.6M [ ] coreutils-6.12.tar.lzma 01-Jun-2008 05:04 3.6M And there's a compressor in ports: archivers/lzma From joerg at britannica.bec.de Tue Nov 25 13:05:32 2008 From: joerg at britannica.bec.de (Joerg Sonnenberger) Date: Tue Nov 25 13:05:39 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <917783.1650.qm@web39102.mail.mud.yahoo.com> References: <917783.1650.qm@web39102.mail.mud.yahoo.com> Message-ID: <20081125210526.GB9737@britannica.bec.de> On Tue, Nov 25, 2008 at 10:56:24AM -0800, bf wrote: > ?? Have you looked at this code? Yes, there is: there is an "LZMA > compressed file format" and the 7z file format, both of which support > LZMA. The former format has been widely adopted by people who distribute > lzma-compressed tarballs, especially GNU-related projects that use the > lzmautils port. Some projects, like GNU coreutils, no longer distribute > the latest versions of their software in bzip2-compressed tarballs. lzma files in the current form are a major PITA. The format doesn't have a proper header and therefore is not really autodetectable. Joerg From bf2006a at yahoo.com Tue Nov 25 12:51:33 2008 From: bf2006a at yahoo.com (bf) Date: Tue Nov 25 13:11:32 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <9bbcef730811251141w63ad793as6efac3e7156bc2ef@mail.gmail.com> Message-ID: <48704.32247.qm@web39107.mail.mud.yahoo.com> --- On Tue, 11/25/08, Ivan Voras wrote: > From: Ivan Voras > Subject: Re: lzma compression/decompression in bsdtar/libarchive? > To: bf2006a@yahoo.com > Cc: freebsd-hackers@freebsd.org > Date: Tuesday, November 25, 2008, 2:41 PM > 2008/11/25 bf : > >> How useful would LZMA be without supporting the > .7z file format? > >> Probably not at all, since there isn't a > gzip-like file format or > >> wrapper that supports LZMA. > > > > ?? Have you looked at this code? Yes, there is: > there is an "LZMA > > compressed file format" and the 7z file format, > both of which support > > LZMA. The former format has been widely adopted by > people who distribute > > lzma-compressed tarballs, especially GNU-related > projects that use the > > lzmautils port. Some projects, like GNU coreutils, no > longer distribute > > the latest versions of their software in > bzip2-compressed tarballs. > > That's interesting - I've never seen an .lzma file > "in the wild". > > But there they are: > http://ftp.gnu.org/gnu/coreutils/ > > [ ] coreutils-6.12.tar.gz 01-Jun-2008 05:03 > 8.6M > [ ] coreutils-6.12.tar.lzma 01-Jun-2008 05:04 > 3.6M > > And there's a compressor in ports: archivers/lzma Yes, a surprising number of projects now give you the option of lzma- compressed tarballs, and have for months. When necessary, they rely on tar to preserve some of the file data you were concerned with, then compress the tarball with lzma, and bundle it in the very simple "lzma compressed file" format, which is roughly: "LZMA compressed file format --------------------------- Offset Size Description 0 1 Special LZMA properties (lc,lp, pb in encoded form) 1 4 Dictionary size (little endian) 5 8 Uncompressed size (little endian). -1 means unknown size 13 Compressed data" as described in the documentation. In the end you obtain compression ratios better than or equal to bzip2 in almost all cases ( usually substantially better), and decompression speeds closer to that of gzip. Compression speed is comparable to, but usually slightly slower than, bzip2. archivers/lzma was the first widely-used implementation, but GNU-inspired projects usually recommend the compatible archivers/lzmautils fork. The benefits can clearly be seen when you compare the size of lzma-compressed tarballs to those using gzip and bzip2. You can see more examples at many of the GNU projects, Graphicsmagick and Imagemagick, etc. -- and many of these are using lzma compression with suboptimal settings. The other night I archived a subversion repository of gentoo portage in a 5.5Mb file by using bsdtar and archivers/lzma. This repository is normally about 420Mb in size, and gentoo's lzma-compressed snapshot tarballs are 29Mb in size, so not all implementations are equal. Not so long ago (the end of April, this year) someone tried to switch ImageMagick to using lzma-compressed tarballs, and caught a lot of flak from others who were unfamiliar with this form of compression. If Tim could integrate it with libarchive, I'm sure that it would be more favorably received. Among the other high-end compression methods, ppmd has attained a stability that would merit support in libarchive, but many of the others are still evolving, or in their present form are too computationally intensive, for diminishing returns, on any but the newest hardware. Regards, b. From rizzo at iet.unipi.it Tue Nov 25 14:42:10 2008 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Tue Nov 25 14:42:19 2008 Subject: convert bootable freebsd or linux iso to bootable flash image In-Reply-To: <20081114131217.GA62275@onelab2.iet.unipi.it> References: <20081114131217.GA62275@onelab2.iet.unipi.it> Message-ID: <20081125224648.GA54158@onelab2.iet.unipi.it> I have updated my iso2flash.sh script so that now it can convert both FreeBSD _and_ linux ISO images to flash images. For the latter, it uses a FAT filesystem, and puts a linux loader with syslinux, for which you can find a port at http://info.iet.unipi.it/~luigi/FreeBSD/index.html#syslinux-port Below is the updated code for iso2flash.sh ------------------- cut here -------------------------------- #!/bin/sh # $Id: iso2flash.sh 897 2008-11-25 22:41:00Z luigi $ # convert an ISO image to flash image # The type of filesystem depends on the content, but can be forced # manually. # # based on picobsd tricks. # Requires makefs, bsdlabel, sed and dd # The linux image uses mtools and syslinux, see # http://info.iet.unipi.it/~luigi/FreeBSD/#syslinux-port # see http://www.allbootdisks.com/download/iso.html MAKEFS=makefs MKLABEL=bsdlabel BSDTAR=tar # Create a linux disk starting from an ISO. Use a FAT media # and syslinux to format it. Add some intelligence to figure # out where the kernel is and what options it needs. make_linux_image() { # src_tree dest_image local tree=$1 local img=$2 local size=$(( $3 + 1000 )) # size in kb local x=$(( 1 + $size / 128 )) # 128k units, good for dd # if you have an old newfs_msdos... local OPTS=" -h 16 -u 64 -S 512 -s $(( 2 * ${size} )) -o 0" [ -f ${img} ] && { chmod u+w ${img}; rm ${img} ; } dd if=/dev/zero bs=128k count=$x of=${img} # create blank file newfs_msdos ${OPTS} ${img} # format msdos syslinux ${img} # add linux bootcode # Try to identify where the kernel is local boot=`find $tree -name boot` if [ x"${boot}" != x -a -d "${boot}" ] ; then echo "moving boot code" chmod u+w ${boot} ${boot}/* mv ${boot}/* $tree [ -f ${tree}/syslinux.cfg ] || mv ${tree}/isolinux.cfg ${tree}/syslinux.cfg fi if [ -d ${tree}/isolinux ] ; then # systemrescuecd local sys=${tree}/syslinux echo moving files... [ -d ${sys} ] || mkdir -p ${sys} chmod -R u+w ${tree}/isolinux mv ${tree}/isolinux/* ${sys} [ -f ${sys}/syslinux.cfg ] || mv ${sys}/isolinux.cfg ${sys}/syslinux.cfg fi if [ -f ${tree}/syslinux.cfg ] ; then elif [ -f ${tree}/syslinux/syslinux.cfg ] ; then elif [ -f ${tree}/linux ] ; then elif [ -f ${tree}/CE_BZ ] ; then # splashtop / expressgate echo "default ce_bz" > ${tree}/syslinux.cfg else boot=`cd ${tree}; find . -name boot.img` if [ x"${boot}" != x -a -f "${tree}/${boot}" ] ; then cp -p /usr/local/share/syslinux/memdisk $tree ( echo "default memdisk"; echo "append initrd=${boot}" ) > $tree/syslinux.cfg fi fi mcopy -i ${img} -s ${tree}/* ::/ # copy the tree mdir -/ -i ${img} :: # show the results } # to add freedos code: #perl sys-freedos.pl --disk=${img} --heads=16 --sectors=64 --offset=0 # --lb #dd if=mbrfat.bin bs=90 iseek=1 oseek=1 of=${img} conv=notrunc # Create a FreeBSD image. make_freebsd_image() { # tree imagefile size local tree=$1 local imagefile=$2 local boot1=${tree}/boot/boot1 local boot2=${tree}/boot/boot2 echo "convert tree $tree image $img" ${MAKEFS} -t ffs -o bsize=4096 -o fsize=512 \ -f 50 ${imagefile} ${tree} ${MKLABEL} -w -f ${imagefile} auto # write a label # copy partition c: into a: with some sed magic ${MKLABEL} -f ${imagefile} | sed -e '/ c:/{p;s/c:/a:/;}' | \ ${MKLABEL} -R -f ${imagefile} /dev/stdin # dump the primary and secondary boot (primary is 512 bytes) dd if=${boot1} of=${imagefile} conv=notrunc 2>/dev/null # XXX secondary starts after the 0x114 = dec 276 bytes of the label # so we skip 276 from the source, and 276+512=788 from dst # the old style blocks used 512 and 1024 respectively dd if=${boot2} iseek=1 ibs=276 2> /dev/null | \ dd of=${imagefile} oseek=1 obs=788 conv=notrunc 2>/dev/null } extract_image() { # extract image to a tree [ -f $1 ] || return local tmp="${tree}.tree" echo "Extract files from ${tree} into $tmp " (chmod -R +w $tmp; rm -rf $tmp ) mkdir -p $tmp ls -la $tmp (cd $tmp && ${BSDTAR} xf $tree ) ls -la $tmp tree=$tmp } guess_type() { echo guess type imgtype="error" # default [ -f $tree/boot/loader -a -f $tree/boot/loader.rc ] && { imgtype="bsd"; return ; } local a=`find $tree -name isolinux` [ x"$a" != x -a -d $a ] && { imgtype="linux"; return ; } } # option processing while [ x"$*" != x ] ; do case x"$1" in x-t ) # type shift imgtype=$1 ;; *) break ;; esac shift done tree=`realpath $1` image=`realpath $2` echo "type <$imgtype> tree <$tree> image <$image>" extract_image $tree set `du -sk $tree` size=$1 echo "image size is $size kb" while true ; do case x"$imgtype" in *[Bb][Ss][Dd] ) make_freebsd_image $tree $image $size ;; *[Ll][Ii][Nn][Uu][Xx] ) make_linux_image $tree $image $size ;; xerror) echo "Image type not found, giving up" ;; * ) guess_type continue ;; esac break done [ -d "$tmp" ] && (chmod -R u+w $tmp && rm -rf $tmp) #---------------------- end of file ------------------------------- From bf2006a at yahoo.com Tue Nov 25 14:30:04 2008 From: bf2006a at yahoo.com (bf) Date: Tue Nov 25 14:56:26 2008 Subject: lzma compression/decompression in bsdtar/libarchive? Message-ID: <511182.46413.qm@web39102.mail.mud.yahoo.com> > lzma files in the current form are a major PITA. The format doesn't have > a proper header and therefore is not really autodetectable. I'm sure that the format has some shortcomings; however, it is being widely used, and so it would be helpful to be able to deal with such files. To be clear: I'm not arguing the merits of any particular *file format*, although I have mentioned a couple of the ones that are now used for lzma-compressed data. (There is no reason why we couldn't define and promote our own file format to contain such data, if we thought that the existing ones didn't meet our needs.) What I am arguing for is to have the capability to use lzma *compression* when it would be to our advantage. Regards, b. From des at des.no Tue Nov 25 16:11:39 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue Nov 25 16:11:46 2008 Subject: Hot swapping SATA drives In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD5E@polaris.maxiscale.com> (Peter Steele's message of "Tue, 25 Nov 2008 08:53:20 -0800") References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD5E@polaris.maxiscale.com> Message-ID: <86hc5vny6e.fsf@ds4.des.no> "Peter Steele" writes: > I've done some searches regarding FreeBSD's 7 support for the hot > swapping of SATA drives and the general consensus appears that it *is* > supported, but not necessarily with all drive models/brands. Make sure the SATA controller is running in AHCI mode. It is disabled by default on most consumer-grade motherboards. DES -- Dag-Erling Sm?rgrav - des@des.no From lists at rhavenn.net Tue Nov 25 17:33:23 2008 From: lists at rhavenn.net (Henrik Hudson) Date: Tue Nov 25 17:33:30 2008 Subject: Hot swapping SATA drives In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD9D@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CD5E@polaris.maxiscale.com> <200811250929.37929.fjwcash@gmail.com> <2ACA3DE8F9758A48B8BE2C7A847F91F240CD9D@polaris.maxiscale.com> Message-ID: <20081126010914.GA30629@alucard.int.rhavenn.net> On Tue, 25 Nov 2008, Peter Steele wrote: > >Use a real hot-swappable drive plane, attached to a good SATA > controller > >that handles hot-swap in hardware? :) > > > >Use ZFS, which seems to work better with drives being added/removed > than > >ata(4)? :) > > > >Sorry, the few systems we have running FreeBSD either have single IDE > >drives, single SATA drives, or 12-24 SATA drives attached to a hardware > > >hot-swappable drive-plane connected to 3Ware 9550/9650 RAID > controllers. > >The single-drive systems obviously can't do swapping, and the rest work > > >without issues. > > I should further clarified that we are running 4-drive systems, with > drive sizes ranging from 250GB-1TB. These drives are not in a RAID > cluster and we do not want them to be. We do need the drives to be hot > swappable though. I'll contact 3Ware and go from there. I would think that if you're using a 3ware controller in JBOD and as stated are in AHCI mode that as long as the OS umounts the drive that you should be able to hot-swap. However, checking with 3ware is probably the best option :) Henrik -- Henrik Hudson lists@rhavenn.net From bu7cher at yandex.ru Tue Nov 25 23:26:52 2008 From: bu7cher at yandex.ru (Andrey V. Elsukov) Date: Tue Nov 25 23:26:59 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: References: <259774.68895.qm@web39105.mail.mud.yahoo.com> <492C3053.1030802@freebsd.org> Message-ID: <492CF763.7090605@yandex.ru> Sean C. Farley wrote: > On Tue, 25 Nov 2008, Tim Kientzle wrote: > >> Where is the announcement of this change? >> >> I haven't downloaded the code yet, but the >> sourceforge project pages all still say GPL. > > It is on the SDK page: http://www.7-zip.org/sdk.html I think you need contact to author. I don't see information about public domain on russian page http://7-zip.org.ua/ru/sdk.html (may be it is translation bug :). LZMA is avaiable under one of three license: 1. GNU LGPL 2. CPL 3. Special author's license (you can use code, but can't change it) -- WBR, Andrey V. Elsukov From stas at FreeBSD.org Wed Nov 26 00:44:16 2008 From: stas at FreeBSD.org (Stanislav Sedov) Date: Wed Nov 26 00:44:24 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <48704.32247.qm@web39107.mail.mud.yahoo.com> References: <9bbcef730811251141w63ad793as6efac3e7156bc2ef@mail.gmail.com> <48704.32247.qm@web39107.mail.mud.yahoo.com> Message-ID: <20081126110435.fe722ef8.stas@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 25 Nov 2008 12:51:33 -0800 (PST) bf mentioned: > Not so long ago (the end of April, this year) someone tried to switch > ImageMagick to using lzma-compressed tarballs, and caught a lot of flak > from others who were unfamiliar with this form of compression. If Tim > could integrate it with libarchive, I'm sure that it would be more > favorably received. > Indeed, the main argument against was that you need an extra dependency to unpack these, and there's no point in requiring that as bz2 versions were available. - -- Stanislav Sedov ST4096-RIPE -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkktAxcACgkQK/VZk+smlYF8DQCfVraD3lSYoZNsPVeYf0tWj9Nr q5UAnjVWWhkSakJWq8K634Pap9hdYDRi =9XOq -----END PGP SIGNATURE----- From bf2006a at yahoo.com Wed Nov 26 01:33:56 2008 From: bf2006a at yahoo.com (bf) Date: Wed Nov 26 04:26:41 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <492CF763.7090605@yandex.ru> Message-ID: <963899.61157.qm@web39103.mail.mud.yahoo.com> --- On Wed, 11/26/08, Andrey V. Elsukov wrote: > From: Andrey V. Elsukov > Subject: Re: lzma compression/decompression in bsdtar/libarchive? > To: "Sean C. Farley" > Cc: "Tim Kientzle" , bf2006a@yahoo.com, freebsd-hackers@FreeBSD.org > Date: Wednesday, November 26, 2008, 2:14 AM > Sean C. Farley wrote: > > On Tue, 25 Nov 2008, Tim Kientzle wrote: > > > >> Where is the announcement of this change? > >> > >> I haven't downloaded the code yet, but the > >> sourceforge project pages all still say GPL. > > > > It is on the SDK page: http://www.7-zip.org/sdk.html > > I think you need contact to author. > I don't see information about public domain on russian > page > http://7-zip.org.ua/ru/sdk.html (may be it is translation > bug :). > LZMA is avaiable under one of three license: > 1. GNU LGPL > 2. CPL > 3. Special author's license (you can use code, but > can't change it) > > -- > WBR, Andrey V. Elsukov If it is a "translation bug", which I think unlikely, it is not just confined to the webpage: the documentation and the copyright notices in the software distribution have all been changed to "public domain". Regards, b. From bf2006a at yahoo.com Wed Nov 26 02:08:41 2008 From: bf2006a at yahoo.com (bf) Date: Wed Nov 26 04:26:53 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <20081126110435.fe722ef8.stas@FreeBSD.org> Message-ID: <29384.68729.qm@web39108.mail.mud.yahoo.com> --- On Wed, 11/26/08, Stanislav Sedov wrote: > From: Stanislav Sedov > Subject: Re: lzma compression/decompression in bsdtar/libarchive? > To: bf2006a@yahoo.com > Cc: "Ivan Voras" , freebsd-hackers@FreeBSD.org > Date: Wednesday, November 26, 2008, 3:04 AM > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 25 Nov 2008 12:51:33 -0800 (PST) > bf mentioned: > > > Not so long ago (the end of April, this year) someone > tried to switch > > ImageMagick to using lzma-compressed tarballs, and > caught a lot of flak > > from others who were unfamiliar with this form of > compression. If Tim > > could integrate it with libarchive, I'm sure that > it would be more > > favorably received. > > > > Indeed, the main argument against was that you need an > extra dependency > to unpack these, and there's no point in requiring that > as bz2 versions > were available. > While I can understand a reluctance to add another port to the dependency tree, on my (slow) machine archivers/lzma occupies 169kb of disk space when installed, and took only a minute or two to build from source, while every time a ImageMagick distfile is downloaded, about 3Mb of network traffic and disk space is saved by using the lzma-compressed distfile rather than the bzip2-compressed distfile. And there are some further time savings in downloading and unpacking. So I think that there is a "point" to switching, even if you didn't agree that it was worthwhile for just one port. When you consider that comparable benefits could have been gained for no further cost for scores of additional ports by switching to lzma, the decision to revert to bzip2 doesn't seem so good. I might add that I have seen ports in the tree that use gzipped tarballs when much smaller bzipped ones are available, and ports that USE_ZIP when there are smaller gzipped tarballs available, so there's more room for improvement. Regards, b. From bf2006a at yahoo.com Wed Nov 26 03:30:13 2008 From: bf2006a at yahoo.com (bf) Date: Wed Nov 26 04:27:07 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <492CF763.7090605@yandex.ru> Message-ID: <164866.53825.qm@web39102.mail.mud.yahoo.com> > > I think you need contact to author. > I don't see information about public domain on russian > page I should add that there has been a discussion of the license change on the project's Sourceforge forums, and the author reaffirms the change: http://sourceforge.net/forum/forum.php?thread_id=2599870&forum_id=45797 Also interesting is the note about adding .zip to the file formats that can be used to support LZMA in version 4.61. Regards, b. From yonyossef.lists at gmail.com Wed Nov 26 05:40:31 2008 From: yonyossef.lists at gmail.com (Yony Yossef) Date: Wed Nov 26 05:40:39 2008 Subject: TSO on VLAN interface, FreeBSD 7 Message-ID: <20def4870811260540x3d44a02bw43b6d18bc675de67@mail.gmail.com> Hi All, When I create a VLAN interface it does not inherit the parent interface capabilities. Is there a way to enable TSO (as other capabilities) on VLAN interfaces? Thanks, Yony From kientzle at freebsd.org Wed Nov 26 10:20:53 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Wed Nov 26 10:20:59 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <164866.53825.qm@web39102.mail.mud.yahoo.com> References: <164866.53825.qm@web39102.mail.mud.yahoo.com> Message-ID: <492D937E.3040000@freebsd.org> bf wrote: >>I think you need contact to author. >>I don't see information about public domain on russian >>page > > I should add that there has been a discussion of the license > change on the project's Sourceforge forums, and the author reaffirms > the change: > > http://sourceforge.net/forum/forum.php?thread_id=2599870&forum_id=45797 Thank you very much for that pointer, that helps a lot. It is a little worrying that Igor Pavlov is asking people whether he should change the license yet again. I hope that's simply rhetorical. Having a license that keeps changing is much worse than any GPL variant. At least it's clear that 4.61 is Public Domain so even if later versions do change, then it should be feasible to fork from there. Maybe over the coming holidays I'll have time to start looking at whether it's feasible to have lzma support in FreeBSD's base system. At least the licensing should no longer be an obstacle, though there could be other hurdles. Tim From tut at nhamon.com.ua Wed Nov 26 10:25:33 2008 From: tut at nhamon.com.ua (Artem Naluzhnyy) Date: Wed Nov 26 10:26:05 2008 Subject: crontab(5) enhancements Message-ID: <65dfa4fc0811261002w61224c0ey9ed987f3f8122db0@mail.gmail.com> Hi, I have two suggestion for newsyslog(8) and periodic(8) entries in default /etc/crontab file: * nice(1) newsyslog and periodic entries to shape CPU load spikes; * lockf(1) them to prevent simultaneous execution. So instead 1 3 * * * root periodic daily there may be 1 3 * * * root lockf -t0 /var/run/periodic.daily.lock nice -20 periodic daily In case of collision we'll receive following email: Date: Wed, 26 Nov 2008 18:16:00 +0200 (EET) From: root@tut.at.home (Cron Daemon) To: root@tut.at.home Subject: Cron lockf -t0 /var/run/periodic.daily.lock nice -20 periodic daily lockf: /var/run/periodic.daily.lock: already locked The configuration has been working for years on my servers without any visible drawbacks. What do you think? -- Artem Naluzhnyy From psteele at maxiscale.com Wed Nov 26 10:46:33 2008 From: psteele at maxiscale.com (Peter Steele) Date: Wed Nov 26 10:46:40 2008 Subject: FreeBSD boot menu is missing Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> I have a procedure for converting a FreeBSD box to use a mirrored slice for the OS. Everything working fine except that after I've made the conversion I am no longer getting the normal boot menu, the one that counts down 10 seconds waiting for the user to pick on option. I see a single line showing that the BTX 1.01 loader has been launched, but from there the system simply boots directly with no menu being displayed. I'm obviously missing a step when using gmirror to convert a system over to use mirroring but I'm not sure what. My basic approach is to install the OS onto the first drive, setting it to use the standard boot manager, and then setup the second drive using gmirror and copy the file systems over to the mirror. I then set boot.config to boot off this drive and it comes up fine, there just isn't any boot menu. Any advice on how to solve this would be appreciated. Thanks. From mwisnicki+freebsd at gmail.com Wed Nov 26 11:04:05 2008 From: mwisnicki+freebsd at gmail.com (Marcin Wisnicki) Date: Wed Nov 26 11:04:12 2008 Subject: lzma compression/decompression in bsdtar/libarchive? References: <259774.68895.qm@web39105.mail.mud.yahoo.com> Message-ID: On Tue, 25 Nov 2008 19:02:02 +0100, Ivan Voras wrote: > > How useful would LZMA be without supporting the .7z file format? > Probably not at all, since there isn't a gzip-like file format or > wrapper that supports LZMA. tar.lzma is quite popular From kientzle at freebsd.org Wed Nov 26 12:03:24 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Wed Nov 26 12:03:34 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: References: <259774.68895.qm@web39105.mail.mud.yahoo.com> Message-ID: <492DAB86.8030500@freebsd.org> Ivan Voras wrote: > bf wrote: >> >>There is good news: Igor Pavlov, the primary author of the original LZMA >>SDK, has placed the latest version, available at: >> >>http://downloads.sourceforge.net/sevenzip/lzma461.tar.bz2 >> >>into the public domain. It's a mix of ANSI-C and C++ code, and so it >>would seem suitable for adoption into the FreeBSD source tree ... > > How useful would LZMA be without supporting the .7z file format? > Probably not at all, since there isn't a gzip-like file format or > wrapper that supports LZMA. Plain .lzma streams are being used today with tar. GNU tar added support recently and read support will be a build option in the upcoming 2.6 release of the libarchive-portable distribution, though it can't be enabled in the FreeBSD base system without the corresponding LZMA libraries. (I expect to get libarchive-portable 2.6 integrated into FreeBSD-CURRENT by the end of the year.) As Joerg pointed out, plain lzma streams are tough to auto-detect, since they only have a 1-byte leading signature. I'm hoping to study the format and find better ways to automatically detect it. Lasse Collin has proposed a wrapper format for LZMA (similar to how gzip format wraps deflate streams) that looks promising, though it's still very new and not yet widely used. I think that's called "lzip". There's also something called "lzop", though I'm not yet sure what exactly that is. The .7z file format is well supported by the 7zip utility in ports, as you pointed out. I don't see any pressing need to support that within FreeBSD base system. Some people are using lzma compression within ZIP archives, and libarchive could easily be extended to support that. (But again, that couldn't be turned on in the FreeBSD base system without the LZMA libraries.) Tim From mwm-keyword-freebsdhackers2.e313df at mired.org Wed Nov 26 12:35:17 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Wed Nov 26 12:35:23 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> Message-ID: <20081126153510.6062cd55@bhuda.mired.org> On Wed, 26 Nov 2008 10:46:29 -0800 "Peter Steele" wrote: > I have a procedure for converting a FreeBSD box to use a mirrored slice > for the OS. Everything working fine except that after I've made the > conversion I am no longer getting the normal boot menu, the one that > counts down 10 seconds waiting for the user to pick on option. > > I see a single line showing that the BTX 1.01 loader has been launched, > but from there the system simply boots directly with no menu being > displayed. I'm obviously missing a step when using gmirror to convert a > system over to use mirroring but I'm not sure what. My basic approach is > to install the OS onto the first drive, setting it to use the standard > boot manager, and then setup the second drive using gmirror and copy the > file systems over to the mirror. I then set boot.config to boot off this > drive and it comes up fine, there just isn't any boot menu. The phrase "and copy the file systems over to the mirror" worries me. Do you actually copy the file systems, or do you let the mirror system do it for you? In particular, are you mirroring file systems or the entire disk? Because the boot blocks aren't part of any file system, so you won't have copied them over, hence you'll be getting whatever boot software the second drive has installed. My recommendation for gmirror is to set up one drive to boot from, then us gmirror label to create a gmirror device on each partition (excluding swap). Edit /etc/fstab to use the gmirror devices thus created, and reboot to make sure it's working properly. It will initially boot from the disk device (pretty much required until gmirror is started), then switch to the mirrored root partition. Now use gmirror insert to add the matching partitions on the second disk, and let gmirror update the bits on the second drive. You'll need to copy the boot blocks from the first drive to the second drive by hand if you want to boot off the second drive. Alternatively, you can mirror the entire drive instead of each partition. That will mirror the boot blocks as well, but means you have to resilver the entire drive instead of the partitions if you have a failure that corrupts a partition. It also makes getting crash dumps more interesting. I did this for a while, but eventually switched to mirroring partitions. FWIW, these days I use ZFS on 64 bit systems in preference to UFS and gmirror. Final comment: if you didn't ask on -questions first, this would have been more appropriate there than here. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From ivoras at freebsd.org Wed Nov 26 12:44:34 2008 From: ivoras at freebsd.org (Ivan Voras) Date: Wed Nov 26 12:44:40 2008 Subject: lzma compression/decompression in bsdtar/libarchive? In-Reply-To: <492DAB86.8030500@freebsd.org> References: <259774.68895.qm@web39105.mail.mud.yahoo.com> <492DAB86.8030500@freebsd.org> Message-ID: <9bbcef730811261244l59b3c7bdq5732af7b9facf1dc@mail.gmail.com> 2008/11/26 Tim Kientzle : > There's also something called "lzop", though I'm not > yet sure what exactly that is. lzop is the LZO archiver - created for high-speed compression, not compression efficiency. Unfortunately, LZO is GPL :( But that reminds me - while you're adding compression formats, you might add lzf (http://www.goof.com/pcg/marc/liblzf.html) which is a very high-speed compressor (again, no emphasis on efficiency), BSDL, implemented in something like 500 lines of C :) "...Very fast compression speeds, rivaling a straight copy loop, especially for decompression which is basically at (unoptimized) memcpy-speed" From psteele at maxiscale.com Wed Nov 26 14:46:50 2008 From: psteele at maxiscale.com (Peter Steele) Date: Wed Nov 26 14:46:58 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <20081126153510.6062cd55@bhuda.mired.org> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <20081126153510.6062cd55@bhuda.mired.org> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> >The phrase "and copy the file systems over to the mirror" worries >me. Do you actually copy the file systems, or do you let the mirror >system do it for you? In particular, are you mirroring file systems or >the entire disk? Because the boot blocks aren't part of any file >system, so you won't have copied them over, hence you'll be getting >whatever boot software the second drive has installed. I'm more or less using the approach described here: http://people.freebsd.org/~rse/mirror/ This assumes you have an existing OS installed on one drive of a multi-drive system. You then use gmirror to create mirror devices on a second drive to match the partitions of the boot drive, transfer the data to the newly established mirror, adjust /etc/fstab on the mirrored root partition to mount the appropriate mirrored devices, then reboot, telling the boot loader to boot from the mirrored drive instead of the original boot drive (via an entry in boot.config). After it comes up, you can then add the original boot drive to the mirror (and any other drive if there are more than two drives that you want to mirror) using gmirror insert. This all works fine, except I'm not getting the boot menu. I know this isn't part of the mirroring, but it is a step I need to perform as part of the whole process. The question is what do I need to do to make sure the appropriate boot loader is setup? >My recommendation for gmirror is to set up one drive to boot from, >then us gmirror label to create a gmirror device on each partition >(excluding swap). Edit /etc/fstab to use the gmirror devices thus >created, and reboot to make sure it's working properly. It will >initially boot from the disk device (pretty much required until >gmirror is started), then switch to the mirrored root partition. Now >use gmirror insert to add the matching partitions on the second disk, >and let gmirror update the bits on the second drive. You'll need to >copy the boot blocks from the first drive to the second drive by hand >if you want to boot off the second drive. I think you are describing more or less the same process here. >FWIW, these days I use ZFS on 64 bit systems in preference to UFS and >gmirror. We plan to switch our application over to ZFS, but not this close to a release. >Final comment: if you didn't ask on -questions first, this would have >been more appropriate there than here. My bad. I'm new in this arena and didn't know where the appropriate place to post. I'm use -questions in the future. From hans at lambermont.dyndns.org Wed Nov 26 14:53:54 2008 From: hans at lambermont.dyndns.org (H) Date: Wed Nov 26 14:54:01 2008 Subject: crontab(5) enhancements In-Reply-To: <65dfa4fc0811261002w61224c0ey9ed987f3f8122db0@mail.gmail.com> References: <65dfa4fc0811261002w61224c0ey9ed987f3f8122db0@mail.gmail.com> Message-ID: <20081126223852.GV23260@frag.lambermont.dyndns.org> Artem Naluzhnyy wrote on 20081126: > I have two suggestion for newsyslog(8) and periodic(8) entries in > default /etc/crontab file: > * nice(1) newsyslog and periodic entries to shape CPU load spikes; > * lockf(1) them to prevent simultaneous execution. I think a pointer to at least lockf will be welcomed by our users. regards, Hans Lambermont From ken at mthelicon.com Wed Nov 26 16:01:32 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Wed Nov 26 16:01:39 2008 Subject: Removing zpool log device - 8.0-Current Message-ID: <200811270001.29378.ken@mthelicon.com> Hello Hackers, I was wondering if anyone knows a work around for this. I have added a log device to one of my zfs pools (it is a unused ATA drive slice that has no activity on it after boot up as most other directories live in there own zfs filing systems). I have added a SSD to the machine and have successfully moved the cache to this device, but wanted to also move the ZIL to another slice on the SSD unit. When I type 'zpool remove PegaBase ar0s1e' I receive the following error: feathers# zpool remove PegaBase ar0s1e cannot remove ar0s1e: only inactive hot spares or cache devices can be removed I know this is a known bug: http://bugs.opensolaris.org/view_bug.do?bug_id=6574286 However, someone mentioned that if you were to flush the ZIL by closing all transactions, you can then remove the device. Does anyone know how this can be done? I have tried the usual fsync, but that dosent seem to flush the zil back to the pool. There seems to be a static amount of information held in the zil and never seems to flush back. Atleast that is how it seems. I would be grateful for any ideas.. Ta Peg From mwm-keyword-freebsdhackers2.e313df at mired.org Wed Nov 26 16:05:52 2008 From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer) Date: Wed Nov 26 16:06:05 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <20081126153510.6062cd55@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> Message-ID: <20081126190545.17b79195@bhuda.mired.org> On Wed, 26 Nov 2008 14:46:44 -0800 "Peter Steele" wrote: > >The phrase "and copy the file systems over to the mirror" worries > >me. Do you actually copy the file systems, or do you let the mirror > >system do it for you? In particular, are you mirroring file systems or > >the entire disk? Because the boot blocks aren't part of any file > >system, so you won't have copied them over, hence you'll be getting > >whatever boot software the second drive has installed. > > I'm more or less using the approach described here: > > http://people.freebsd.org/~rse/mirror/ > > This assumes you have an existing OS installed on one drive of a > multi-drive system. You then use gmirror to create mirror devices on a > second drive to match the partitions of the boot drive, transfer the > data to the newly established mirror, adjust /etc/fstab on the mirrored > root partition to mount the appropriate mirrored devices, then reboot, > telling the boot loader to boot from the mirrored drive instead of the > original boot drive (via an entry in boot.config). After it comes up, > you can then add the original boot drive to the mirror (and any other > drive if there are more than two drives that you want to mirror) using > gmirror insert. This all works fine, except I'm not getting the boot > menu. I know this isn't part of the mirroring, but it is a step I need > to perform as part of the whole process. The question is what do I need > to do to make sure the appropriate boot loader is setup? He had you install a stock MBR on the second disk. You never copied the boot loader from the first disk, so that's what you're going to use when you boot from the second disk. You need to install the boot block you want on the second disk. Which probably means boot0. boot0cfg will do that for you. You probably want boot0cfg -B -s 1 # The device - ad1, not the slice! > >My recommendation for gmirror is to set up one drive to boot from, > >then us gmirror label to create a gmirror device on each partition > >(excluding swap). Edit /etc/fstab to use the gmirror devices thus > >created, and reboot to make sure it's working properly. It will > >initially boot from the disk device (pretty much required until > >gmirror is started), then switch to the mirrored root partition. Now > >use gmirror insert to add the matching partitions on the second disk, > >and let gmirror update the bits on the second drive. You'll need to > >copy the boot blocks from the first drive to the second drive by hand > >if you want to boot off the second drive. > I think you are describing more or less the same process here. Um, no. He reduced the size of one partition because he's overly paranoid about gmirror failing to recognize the providers properly, which forces him to dump and restore one partition - which leads to doing them all to get them on one disk. If you don't need to resize the partitions, you can just labelling the disk you're already using. Once you've done that, you can gmirror insert the second drive into the mirror, and it will resilver the second drive while providing full access to the first one. No need to copy any data at all. His analysis of the choices is pretty shallow as well. He lets wanting to use different-sized disks dominate the analysis, which is great if you're building your mirror with disks from the parts bin. I tend to by drives to pairs if I want to mirror them, so that's immaterial. Once that's gone, mirroring a full disk slice just doesn't make sense at all - either mirror the entire disk (to get the MBR), or mirror the partitions in the slice (for extra flexibility and less painful resilvering). Better instructions for getting a full-disk mirror can be found here: http://www.onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From psteele at maxiscale.com Wed Nov 26 16:56:06 2008 From: psteele at maxiscale.com (Peter Steele) Date: Wed Nov 26 16:56:13 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <20081126190545.17b79195@bhuda.mired.org> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com><20081126153510.6062cd55@bhuda.mired.org><2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> >He had you install a stock MBR on the second disk. You never copied >the boot loader from the first disk, so that's what you're going to >use when you boot from the second disk. You need to install the boot >block you want on the second disk. Which probably means >boot0. boot0cfg will do that for you. You probably want > boot0cfg -B -s 1 # The device - ad1, not the slice! Okay, that makes sense. That's an easy change to my script. >Um, no. He reduced the size of one partition because he's overly >paranoid about gmirror failing to recognize the providers properly, >which forces him to dump and restore one partition - which leads to >doing them all to get them on one disk. If you don't need to resize >the partitions, you can just labelling the disk you're already using. >Once you've done that, you can gmirror insert the second drive into >the mirror, and it will resilver the second drive while providing full >access to the first one. No need to copy any data at all. Man, I wish I'd known this. I built a whole automated framework around this, assuming you couldn't set up the initial mirror drive with a live file system. I'll have to try your solution; it is definitely the way to go. We are dealing with identical size drives as well so this shouldn't be a problem. >His analysis of the choices is pretty shallow as well. He lets wanting >to use different-sized disks dominate the analysis, which is great if >you're building your mirror with disks from the parts bin. I tend to >by drives to pairs if I want to mirror them, so that's >immaterial. Once that's gone, mirroring a full disk slice just doesn't >make sense at all - either mirror the entire disk (to get the MBR), or >mirror the partitions in the slice (for extra flexibility and less >painful resilvering). We don't want to mirror the whole drive, just the OS partitions. I decided to go with the full slice mirroring because of what was described in this link. If mirroring the partitions in the slice is the better way to go, then that's fine my me. >Better instructions for getting a full-disk mirror can be found here: >http://www.onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html I look forward to reading this. Thanks for the help! From ancelgray at yahoo.com Wed Nov 26 19:49:43 2008 From: ancelgray at yahoo.com (ancelgray) Date: Wed Nov 26 21:00:49 2008 Subject: Hardware support for AMD Geode CS5536 audio? In-Reply-To: <20080121170155.GC51116@hamlet.SetFilePointer.com> References: <20080121170155.GC51116@hamlet.SetFilePointer.com> Message-ID: <20713056.post@talk.nabble.com> To AMD CS5536 users: This is Andrew Gray. I have finished the audio driver for the AMD CS5536 companion chip. It is working on a PC Engines Alix 1C low power board under FreeBSD 7.0. It can be found at: http://modelofreality.org/snd_amd5536.html Let me know how it goes. Andrew Gray ancelgray AT y a h o o DOT com -- View this message in context: http://www.nabble.com/Hardware-support-for-AMD-Geode-CS5536-audio--tp15002428p20713056.html Sent from the freebsd-hackers mailing list archive at Nabble.com. From eitanadlerlist at gmail.com Wed Nov 26 21:46:19 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Wed Nov 26 21:46:25 2008 Subject: `type' built-in command in /bin/sh In-Reply-To: <20081124084016.GA1384@lizard.fafoe.narf.at> References: <87vdueavft.fsf@h1.lan> <20081124084016.GA1384@lizard.fafoe.narf.at> Message-ID: <492E3424.4080109@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Stefan Farfeleder wrote: > On Mon, Nov 24, 2008 at 02:12:22AM +0300, Zajcev Evgeny wrote: >> Hello there. I just hit into "problem" that built-in sh command type >> outputs to stdout when error occurs. I mean it returns non-0 status, >> but outputs to stdout! I think stderr has been made exactly for such >> purposes and `type' should use it in case of error .. what do you >> think? As far as I could tell if (cmd != TYPECMD_SMALLV) out1str(": not found\n"); has to be changed to if (cmd != TYPECMD_SMALLV) out2str(": not found\n"); C is not my strongest language so please let me know if I am correct. - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkuNCQACgkQtl8kq+nCzNGrGgCfZu5MHngEnMyQ5X+g1ZtKraJu blsAn3xoReD4+mVYU5Agw3Ns5CyIIO+p =0cXm -----END PGP SIGNATURE----- From des at des.no Thu Nov 27 02:45:12 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Thu Nov 27 02:45:18 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> (Peter Steele's message of "Wed, 26 Nov 2008 16:53:11 -0800") References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <20081126153510.6062cd55@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> Message-ID: <861vwx4fd5.fsf@ds4.des.no> "Peter Steele" writes: > Man, I wish I'd known this. I built a whole automated framework around > this, assuming you couldn't set up the initial mirror drive with a live > file system. I'll have to try your solution; it is definitely the way to > go. We are dealing with identical size drives as well so this shouldn't > be a problem. Just make sure you leave a few unallocated blocks at the end of the disk (for gmirror metadata). In most cases, this happens automatically, because the size of the disk is not a multiple of the (fake) cylinder size. To pick one at random, I have a Maxtor 6B300S0 here with 586112591 LBA sectors where the fake c/h/s geometry only adds up to 586111680 sectors, leaving 911 sectors at the end. > We don't want to mirror the whole drive, just the OS partitions. I > decided to go with the full slice mirroring because of what was > described in this link. If mirroring the partitions in the slice is the > better way to go, then that's fine my me. Mirroring the entire slice is far simpler. If you mirror individual partitions, you have to label them *before* you newfs them. I would mirror the whole drive, though - and I would use ZFS, with which you can easily transition to larger drives (just replace them one by one and resilver in between - you can even do it online if your disks are hot-swappable) DES -- Dag-Erling Sm?rgrav - des@des.no From avg at icyb.net.ua Thu Nov 27 04:40:15 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Thu Nov 27 04:40:26 2008 Subject: ukbd attachment and root mount In-Reply-To: <491ADB3B.2090000@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> <491AD0CB.8050309@icyb.net.ua> <20081112132124.GA25637@icarus.home.lan> <491ADB3B.2090000@icyb.net.ua> Message-ID: <492E952A.80104@icyb.net.ua> Setting this issue on a technical trail now. 1. I built a kernel with USB_DEBUG enabled. BTW, there doesn't seem to be a way to set debug levels for USB subsystems at boot time, i.e. via hints. Or am I missing something? It seems that the levels can only be set via sysctl but that's too late for boot time debugging. I had to hardcode some non-zero initial values for the levels. 2. I performed a verbose boot with USB_DEBUG kernel. 3. I looked through the dmesg and through code. Some observations and thoughts. There seem to be 3 points where devices attached via USB get explored/discovered and probed/matched/attached. First of all, typical USB controllers are attached to PCI, so ehci/uhci/ohci devices and their corresponding usb and uhub devices are attached along with other PCI devices. Then, for EHCI usb devices bus exploration is performed immediately and so some devices can get attached quite early (e.g. umass). This is the first point. For UHCI/OHCI hubs are added to special cold exploration list. And also event threads are created for all hubs. Then, via SYSINIT mechanism buses in the "cold list" get explored. Actual priority is SI_SUB_CONFIGURE:SI_ORDER_MIDDLE. This is the second point. And finally the event threads get executed and after some delay (about 4 seconds) they also explore their buses. What I observe here matched the described behavior but only to a certain extent: 1. I see that certain devices like an external USB hub get reported in dmesg among PCI devices. I understand that this is the first point ("ehci"). 2. My USB mouse (low speed, attached to uhci) gets reported somewhere between the following lines: isa_probe_children: probing PnP devices ... Device configuration finished. I understand that this is the second point ("sysinit"). 3. My USB keyboard gets reported after mountroot (but before start of init). I think that this is the third point. So what is very puzzling to me is why the keyboard is not found along with the mouse at the second point. Especially given that they are attached to the ports of the same hub: Controller /dev/usb2: addr 1: full speed, self powered, config 1, UHCI root hub(0x0000), Intel(0x0000), rev 1.00 uhub2 port 1 addr 2: low speed, power 98 mA, config 1, USB-PS/2 Optical Mouse(0xc040), Logitech(0x046d), rev 24.30 ums0 port 2 addr 3: low speed, power 100 mA, config 1, USB Keyboard(0x0101), CHESEN(0x0a81), rev 1.10 ukbd0 uhid0 Here's a snippet from verbose dmesg wth USB_DEBUG where the mouse is reported: isa_probe_children: probing PnP devices uhub_explore: status change hub=1 port=1 usbd_reset_port: port 1 reset done, error=NORMAL_COMPLETION usbd_new_device bus=0xffffffff80c7d000 port=1 depth=1 speed=1 usbd_setup_pipe: dev=0xffffff0004a16d00 iface=0 ep=0xffffff0004a16d38 pipe=0xffffff0004a16d08 uhci_open: pipe=0xffffff0004a16c00, addr=0, endpt=0 (1) usb_allocmem: adding fragments usbd_new_device: adding unit addr=2, rev=200, class=0, subclass=0, protocol=0, maxpacket=8, len=18, speed=1 usbd_ar_pipe: pipe=0xffffff0004a16c00 usbd_setup_pipe: dev=0xffffff0004a16d00 iface=0 ep=0xffffff0004a16d38 pipe=0xffffff0004a16d08 uhci_open: pipe=0xffffff0004a16b00, addr=0, endpt=0 (1) usbd_ar_pipe: pipe=0xffffff0004a16b00 usbd_setup_pipe: dev=0xffffff0004a16d00 iface=0 ep=0xffffff0004a16d38 pipe=0xffffff0004a16d08 uhci_open: pipe=0xffffff0004a16a00, addr=2, endpt=0 (1) usbd_new_device: new dev (addr 2), dev=0xffffff0004a16d00, parent=0xffffff0001338c00 usbd_probe_and_attach: trying device specific drivers usbd_probe_and_attach: no device specific driver found usbd_probe_and_attach: looping over 1 configurations usbd_probe_and_attach: trying config idx=0 usbd_set_config_index: (addr 1) cno=2 attr=0xa0, selfpowered=0, power=98 usbd_set_config_index: set config 1 ums0: on uhub2 ums0: 8 buttons and Z dir. uhub_explore: status change hub=1 port=2 uhub_explore: status change hub=1 port=1 Device configuration finished. And here's how the keyboard is found later: Trying to mount root from zfs:tank/root usbd_new_device bus=0xffffffff80c7d000 port=2 depth=1 speed=1 usbd_setup_pipe: dev=0xffffff0004b53000 iface=0 ep=0xffffff0004b53038 pipe=0xffffff0004b53008 uhci_open: pipe=0xffffff0001242c00, addr=0, endpt=0 (1) usbd_new_device: adding unit addr=3, rev=110, class=0, subclass=0, protocol=0, maxpacket=8, len=18, speed=1 usbd_ar_pipe: pipe=0xffffff0001242c00 usbd_setup_pipe: dev=0xffffff0004b53000 iface=0 ep=0xffffff0004b53038 pipe=0xffffff0004b53008 uhci_open: pipe=0xffffff0001242e00, addr=0, endpt=0 (1) usbd_ar_pipe: pipe=0xffffff0001242e00 usbd_setup_pipe: dev=0xffffff0004b53000 iface=0 ep=0xffffff0004b53038 pipe=0xffffff0004b53008 uhci_open: pipe=0xffffff0004b53100, addr=3, endpt=0 (1) usbd_new_device: new dev (addr 3), dev=0xffffff0004b53000, parent=0xffffff0001338c00 usbd_probe_and_attach: trying device specific drivers usbd_probe_and_attach: no device specific driver found usbd_probe_and_attach: looping over 1 configurations usbd_probe_and_attach: trying config idx=0 usbd_set_config_index: (addr 1) cno=3 attr=0xa0, selfpowered=0, power=100 usbd_set_config_index: set config 1 ukbd0: on uhub2 Full dmesg is here: http://www.icyb.net.ua/~avg/ukbd.dmesg.gz -- Andriy Gapon From yonyossef.lists at gmail.com Thu Nov 27 04:41:38 2008 From: yonyossef.lists at gmail.com (Yony Yossef) Date: Thu Nov 27 04:41:50 2008 Subject: IFCAP_LRO on FreeBSD 7 Message-ID: <20def4870811270441n6b201076k715e1c7e1adc8781@mail.gmail.com> Hi All, Is there a native interface for LRO in FreeBSD 7? I can't find any use for IFCAP_LRO but notifying the driver if to use or not to use this offload. If not, is it planned for FreeBSD 8? Thanks Yony From rwatson at FreeBSD.org Thu Nov 27 04:51:29 2008 From: rwatson at FreeBSD.org (Robert Watson) Date: Thu Nov 27 04:51:45 2008 Subject: IFCAP_LRO on FreeBSD 7 In-Reply-To: <20def4870811270441n6b201076k715e1c7e1adc8781@mail.gmail.com> References: <20def4870811270441n6b201076k715e1c7e1adc8781@mail.gmail.com> Message-ID: On Thu, 27 Nov 2008, Yony Yossef wrote: > Is there a native interface for LRO in FreeBSD 7? I can't find any use for > IFCAP_LRO but notifying the driver if to use or not to use this offload. > > If not, is it planned for FreeBSD 8? IFCAP_LRO is a capability/policy flag allowing drivers to declare support for LRO, and for administrators to enable/disable it if present. Drivers can either provide their own implementation (mxge, nxge) or use the system implementation (cxgb, igb). I'm slightly surprised to see that igb references tcp_lro_init() but not IFCAP_LRO -- perhaps lro isn't yet fully hooked up, or perhaps there's a bug? I believe all of the above applies to 7.1 but not 7.0, except possibly mxge supporting LRO in 7.0. Robert N M Watson Computer Laboratory University of Cambridge From avg at icyb.net.ua Thu Nov 27 05:23:21 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Thu Nov 27 05:23:27 2008 Subject: ukbd attachment and root mount In-Reply-To: <492E952A.80104@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> <491AD0CB.8050309@icyb.net.ua> <20081112132124.GA25637@icarus.home.lan> <491ADB3B.2090000@icyb.net.ua> <492E952A.80104@icyb.net.ua> Message-ID: <492E9F45.2090404@icyb.net.ua> I increased debug level in uhub and also switched mouse and keyboard ports hoping that order might matter. It didn't. Here's fresh usbdevs output snippet: Controller /dev/usb2: addr 1: full speed, self powered, config 1, UHCI root hub(0x0000), Intel(0x0000), rev 1.00 uhub2 port 1 addr 3: low speed, power 100 mA, config 1, USB Keyboard(0x0101), CHESEN(0x0a81), rev 1.10 ukbd0 uhid0 port 2 addr 2: low speed, power 98 mA, config 1, USB-PS/2 Optical Mouse(0xc040), Logitech(0x046d), rev 24.30 ums0 And here's a new snippet from cold explore dmesg: uhub2: uhub_explore: port 1 status 0x0100 0x0001 + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + So, hm, it looks like a change in connection status is reported but current status is reported as not connected. + I wonder why? + Could this be related to how we perform UHCI handover from BIOS to kernel? + Our uhci code seems to be much simpler than what MS folks described here: + http://www.microsoft.com/whdc/archive/usbhost.mspx#EQHAC uhub_explore: status change hub=1 port=1 uhub_explore: port=1 !CURRENT_CONNECT_STATUS uhub2: uhub_explore: port 2 status 0x0301 0x0001 uhub_explore: status change hub=1 port=2 usbd_reset_port: port 2 reset done, error=NORMAL_COMPLETION usbd_new_device bus=0xffffffff80c7d000 port=2 depth=1 speed=1 usbd_setup_pipe: dev=0xffffff0004a16d00 iface=0 ep=0xffffff0004a16d38 pipe=0xffffff0004a16d08 uhci_open: pipe=0xffffff0004a16c00, addr=0, endpt=0 (1) usb_allocmem: adding fragments usbd_new_device: adding unit addr=2, rev=200, class=0, subclass=0, protocol=0, maxpacket=8, len=18, speed=1 usbd_ar_pipe: pipe=0xffffff0004a16c00 usbd_setup_pipe: dev=0xffffff0004a16d00 iface=0 ep=0xffffff0004a16d38 pipe=0xffffff0004a16d08 uhci_open: pipe=0xffffff0004a16b00, addr=0, endpt=0 (1) usbd_ar_pipe: pipe=0xffffff0004a16b00 usbd_setup_pipe: dev=0xffffff0004a16d00 iface=0 ep=0xffffff0004a16d38 pipe=0xffffff0004a16d08 uhci_open: pipe=0xffffff0004a16a00, addr=2, endpt=0 (1) usbd_new_device: new dev (addr 2), dev=0xffffff0004a16d00, parent=0xffffff0001338c00 usbd_probe_and_attach: trying device specific drivers usbd_probe_and_attach: no device specific driver found usbd_probe_and_attach: looping over 1 configurations usbd_probe_and_attach: trying config idx=0 usbd_set_config_index: (addr 1) cno=2 attr=0xa0, selfpowered=0, power=98 usbd_set_config_index: set config 1 ums0: on uhub2 ums0: 8 buttons and Z dir. -- Andriy Gapon From psteele at maxiscale.com Thu Nov 27 08:19:42 2008 From: psteele at maxiscale.com (Peter Steele) Date: Thu Nov 27 08:19:48 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <861vwx4fd5.fsf@ds4.des.no> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com><20081126153510.6062cd55@bhuda.mired.org><2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com><20081126190545.17b79195@bhuda.mired.org><2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> >Mirroring the entire slice is far simpler. If you mirror individual >partitions, you have to label them *before* you newfs them. What we're really trying to accomplish is an automated install via a PXE boot server. Unfortunately gmirror isn't available in mfsroot at the point the file systems need to be set up. So what we've ended up doing is doing is what amounts to a bootstrap install on the first disk, and then after the installCommit is done, gmirror is available and we have a post install script that runs gmirror on the other drives. Then the script copies the OS slice over to the gmirrored fs, reboots to this mirrored system, and finally adds the original disk to the mirror. It's fully automated and gives us a mirrored OS slice across four drives, and we even handle drives of different sizes. >I would mirror the whole drive, though We can't do that. The data on the non-mirrored portion is different on each drive and we don't want it mirrored. > - and I would use ZFS, with which >you can easily transition to larger drives (just replace them one by one >and resilver in between - you can even do it online if your disks are >hot-swappable) FreeBSD doesn't handle hot swap very well we've discovered, not unless you are using a RAID based backplane and drives. We cannot use RAID in our application, and don't in fact want to. We're still trying to figure out how to deal with drive removal in a live non-RAIDed system. We plan to move to ZFS but we are too close to a release cycle to make the move now (QA would have to run through weeks of testing). ZFS will happen, though, sooner or later. From des at des.no Thu Nov 27 08:49:48 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Thu Nov 27 08:49:55 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> (Peter Steele's message of "Thu, 27 Nov 2008 08:19:18 -0800") References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <20081126153510.6062cd55@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> Message-ID: <86myfl15cl.fsf@ds4.des.no> "Peter Steele" writes: > Dag-Erling Sm?rgrav writes: > > Mirroring the entire slice is far simpler. If you mirror individual > > partitions, you have to label them *before* you newfs them. > What we're really trying to accomplish is an automated install via a > PXE boot server. Unfortunately gmirror isn't available in mfsroot at > the point the file systems need to be set up. So what we've ended up > doing is doing is what amounts to a bootstrap install on the first > disk, and then after the installCommit is done, gmirror is available > and we have a post install script that runs gmirror on the other > drives. Then the script copies the OS slice over to the gmirrored fs, > reboots to this mirrored system, and finally adds the original disk to > the mirror. It's fully automated and gives us a mirrored OS slice > across four drives, and we even handle drives of different sizes. Exactly. So what you do, instead, is make sure there is a little space left over at the end of the slice that you create in the first step. Then, once gmirror is available, you gmirror label the slice, then gmirror insert the corresponding slice on the other disk(s), and gmirror rebuild. No copying involved; gmirror takes care of it all. The key here is that 'gmirror label' is non-destructive as long as the last sector on the provider is unused. > FreeBSD doesn't handle hot swap very well we've discovered, not unless > you are using a RAID based backplane and drives. It does, AFAIK, even on SATA, provided the controller supports it and is configured correctly. DES -- Dag-Erling Sm?rgrav - des@des.no From psteele at maxiscale.com Thu Nov 27 10:54:51 2008 From: psteele at maxiscale.com (Peter Steele) Date: Thu Nov 27 10:55:01 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <86myfl15cl.fsf@ds4.des.no> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com><20081126153510.6062cd55@bhuda.mired.org><2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com><20081126190545.17b79195@bhuda.mired.org><2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com><861vwx4fd5.fsf@ds4.des.no><2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> <86myfl15cl.fsf@ds4.des.no> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> >So what you do, instead, is make sure there is a little space left over >at the end of the slice that you create in the first step. Then, once >gmirror is available, you gmirror label the slice, then gmirror insert >the corresponding slice on the other disk(s), and gmirror rebuild. No >copying involved; gmirror takes care of it all. > >The key here is that 'gmirror label' is non-destructive as long as the >last sector on the provider is unused. The problem is I was unable to get multiple slices defined in a sysinstall config script. I tried many variations of parameters to pump into diskPartitionEditor and diskLabelEditor so that we could create three slices during the install but I couldn't find anything that worked. So I ended up having to create a single full disk slice to install the OS onto, and then in a post commit step slice the disks up as we want them and copy the OS over. I couldn't find a single example how to create multiple slices in a sysinstall config file. If you know how to do this, I'd love to see it. >It does, AFAIK, even on SATA, provided the controller supports it and is >configured correctly. With the proper controller and drive, yes, FreeBSD does support hot swap, to a point. Let's say for example that you have a file system mounted on a drive and that drive dies. You can pull it and put in a new one, but FreeBSD will not let you unmount the file system on the original drive. Even umount -f fails. We have to reboot to get the old mount point released, and we haven't found any way around this. From des at des.no Thu Nov 27 11:44:33 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Thu Nov 27 11:44:41 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> (Peter Steele's message of "Thu, 27 Nov 2008 10:54:31 -0800") References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <20081126153510.6062cd55@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> <86myfl15cl.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> Message-ID: <86wsepymw0.fsf@ds4.des.no> "Peter Steele" writes: > The problem is I was unable to get multiple slices defined in a > sysinstall config script. I tried many variations of parameters to > pump into diskPartitionEditor and diskLabelEditor so that we could > create three slices during the install but I couldn't find anything > that worked. So I ended up having to create a single full disk slice > to install the OS onto, and then in a post commit step slice the disks > up as we want them and copy the OS over. I couldn't find a single > example how to create multiple slices in a sysinstall config file. If > you know how to do this, I'd love to see it. I wouldn't use a sysinstall script. Set up a file system (say /nfsroot) on an NFS server in your lab. Install FreeBSD on /nfsroot: either make installworld DESTDIR=/nfsroot followed by the correct mergemaster incantation, or just run the install.sh script in the base and kernels directories: # cd /mnt/7.0-RELEASE/base # env DESTDIR=/nfsroot sh install.sh # cd ../kernels # env DESTDIR=/nfsroot sh install.sh GENERIC Put a complete copy of the release CD somewhere in /nfsroot so you can access it from the client later. Make sure /nfsroot is listed in /etc/exports with the correct permissions (showmount -e is your friend) Copy /nfsroot/boot/pxeboot to /tftpboot/pxeboot and enable tftpd. Set up a DHCP server on your lab subnet or whatever, with next-server set to point at your NFS+TFTP server, filename set to "pxeboot", and option root-path set to /nfsroot. Replace /nfsroot/etc/rc with a shell script that performs the installation just the way you like it. Your script should probably start by setting up memory filesystems for /tmp and /var, otherwise stuff like disklabel might not work. Connect the new box to the lab network, turn it on, enable PXE, and let it boot. If you don't want to use PXE + NFS, you can create an installation CD with almost the exact same procedure: create a directory, follow the above procedure to populate it (just ignore the parts about NFS, TFTP and DHCP), create a bootable ISO image of that directory, burn it to a DVD, and boot the box from that DVD. The correct mkisofs incantation is: mkisofs -JR -b /cdroot/boot/cdboot -o cd.iso /cdroot If you can't use a DVD, you'll have to leave stuff out - packages and the ports and src distributions are obvious candidates. A full i386 7.0R live CD with a copy of the installation CD minus packages and src should fit on a regular CD. > With the proper controller and drive, yes, FreeBSD does support hot > swap, to a point. Let's say for example that you have a file system > mounted on a drive and that drive dies. You can pull it and put in a > new one, but FreeBSD will not let you unmount the file system on the > original drive. Even umount -f fails. We have to reboot to get the old > mount point released, and we haven't found any way around this. That's an entirely different matter... that's why you use gmirror or graid or zfs or whatever, so you can swap out the drive online. DES -- Dag-Erling Sm?rgrav - des@des.no From psteele at maxiscale.com Thu Nov 27 11:55:54 2008 From: psteele at maxiscale.com (Peter Steele) Date: Thu Nov 27 11:56:00 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <86wsepymw0.fsf@ds4.des.no> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com><20081126153510.6062cd55@bhuda.mired.org><2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com><20081126190545.17b79195@bhuda.mired.org><2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com><861vwx4fd5.fsf@ds4.des.no><2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com><86myfl15cl.fsf@ds4.des.no><2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> <86wsepymw0.fsf@ds4.des.no> Message-ID: <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC6@polaris.maxiscale.com> >I wouldn't use a sysinstall script. Yeah, I should probably have done it that way but I inherited the existing sysinstall framework from someone else and ended up extending it to use gmirror. I know more about this area now and I'd like to redo the whole thing, avoiding sysinstall. That will have to be a future project though. >That's an entirely different matter... that's why you use gmirror or >graid or zfs or whatever, so you can swap out the drive online. RAID is not an option for us, at least not for this particular problem. Long story. From kevin at your.org Thu Nov 27 12:03:55 2008 From: kevin at your.org (Kevin Day) Date: Thu Nov 27 12:04:03 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <86wsepymw0.fsf@ds4.des.no> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <20081126153510.6062cd55@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> <86myfl15cl.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> <86wsepymw0.fsf@ds4.des.no> Message-ID: On Nov 27, 2008, at 1:44 PM, Dag-Erling Sm?rgrav wrote: > > I wouldn't use a sysinstall script. > > Set up a file system (say /nfsroot) on an NFS server in your lab. Just in case anyone needs a real step-by-step guide to getting a diskless PXE/NFS boot going, I wrote this up a little while ago. http://sigsegv.org/wiki/How_to_do_a_PXE_netboot_install_of_FreeBSD The existing documentation was either a bit out of date, glossed over some of it or just flat out didn't work for me. Takes an existing running system, clones it into /nfsroot, enables dhcpd/tftpd/nfsd, and relies on rc.diskless to make the magic happen after boot. From jfvogel at gmail.com Thu Nov 27 12:11:36 2008 From: jfvogel at gmail.com (Jack Vogel) Date: Thu Nov 27 12:25:39 2008 Subject: IFCAP_LRO on FreeBSD 7 In-Reply-To: References: <20def4870811270441n6b201076k715e1c7e1adc8781@mail.gmail.com> Message-ID: <2a41acea0811271143q25f83e89i4df2791052a980ac@mail.gmail.com> On Thu, Nov 27, 2008 at 4:51 AM, Robert Watson wrote: > > On Thu, 27 Nov 2008, Yony Yossef wrote: > > Is there a native interface for LRO in FreeBSD 7? I can't find any use for >> IFCAP_LRO but notifying the driver if to use or not to use this offload. >> >> If not, is it planned for FreeBSD 8? >> > > IFCAP_LRO is a capability/policy flag allowing drivers to declare support > for LRO, and for administrators to enable/disable it if present. Drivers > can either provide their own implementation (mxge, nxge) or use the system > implementation (cxgb, igb). I'm slightly surprised to see that igb > references tcp_lro_init() but not IFCAP_LRO -- perhaps lro isn't yet fully > hooked up, or perhaps there's a bug? I believe all of the above applies to > 7.1 but not 7.0, except possibly mxge supporting LRO in 7.0. > Been so busy internally that I did not even realize that this capability had been created, I need to change both igb and ixgbe to use it. Thanks for pointing this out, Jack From yanefbsd at gmail.com Thu Nov 27 12:30:05 2008 From: yanefbsd at gmail.com (Garrett Cooper) Date: Thu Nov 27 12:30:12 2008 Subject: FreeBSD boot menu is missing In-Reply-To: References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> <86myfl15cl.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> <86wsepymw0.fsf@ds4.des.no> Message-ID: <7d6fde3d0811271230p512d1325i591e9711caa432d8@mail.gmail.com> On Thu, Nov 27, 2008 at 12:03 PM, Kevin Day wrote: > > On Nov 27, 2008, at 1:44 PM, Dag-Erling Sm?rgrav wrote: >> >> I wouldn't use a sysinstall script. >> >> Set up a file system (say /nfsroot) on an NFS server in your lab. > > > Just in case anyone needs a real step-by-step guide to getting a diskless > PXE/NFS boot going, I wrote this up a little while ago. > > http://sigsegv.org/wiki/How_to_do_a_PXE_netboot_install_of_FreeBSD > > The existing documentation was either a bit out of date, glossed over some > of it or just flat out didn't work for me. Takes an existing running system, > clones it into /nfsroot, enables dhcpd/tftpd/nfsd, and relies on rc.diskless > to make the magic happen after boot. That's a good brief guide, but a lot of the information is already contained in: . That reminds me that I need to add some documentation for booting OpenFirmware Mac's with Netboot (some experiences that I had at Cisco that I wanted to pass down for others). -Garrett From christoph.mallon at gmx.de Thu Nov 27 13:06:29 2008 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Thu Nov 27 13:06:38 2008 Subject: New C compiler and analyzer lang/cparser in ports Message-ID: <492F0591.7050807@gmx.de> A few days ago libFIRM[1] and cparser were added to the ports tree. If you want to see, what other compilers besides GCC have to offer, this might be of interest for you. libFIRM is a modern optimizing intermediate representation (IR) library. cparser is a C compiler providing many useful warnings and uses libFIRM for optimization and code generation. libFIRM is a library implementing a modern graph based IR for compilers, which exclusively operates on program graphs in SSA-form. It is developed at the IPD at the Universit?t Karlsruhe (TH)[2]. SSA is maintained beginning at the construction of the IR graph until assembler code is emitted. libFIRM offers many analyses and optimizations, provides extensive debugging support and includes a backend framework. At the moment it can produce code for 32bit x86 processors. There are also unfinished backends for ARM, MIPS and PPC32. The backend uses a novel SSA based register allocator and includes several algorithms for instruction scheduling, spilling, copy coalescing and basic block scheduling. All stages of the compilation process can be dumped and inspected with our graph viewer yComp[3]. cparser is a C compiler, which can parse C89 and C99 as well as many GCC and some MSVC extensions. The handled GCC extensions include __attribute__, inline assembler, computed goto and statement expressions. The compiler driver is compatible with with GCC (-fxxx, -Wxxx, -M, ...). It also provides many useful analyses for warnings - for some examples see below. Take a look at the examples below to get a glimpse of what cparser and libFIRM have to offer. You can find more information about them at http://www.libfirm.org. Of course, they are capable of compiling non-trivial programs like the SPEC INT2000 suite and Quake 3[4]. If I have sparked your interest in cparser and libFIRM, try the port lang/cparser. If you have any questions, suggestions or problems regarding cparser and libFIRM, please feel free to contact us. Regards Christoph Examples -------- One goal is to provide concise error messages, which help to pinpoint the cause of the problem. This is achieved by using a handwritten recursive descent parser, which includes some heuristics to produce sensible error messages. A further aim is to restart parsing immediately after a parse error, in order to avoid subsequent errors. In this way, the number of stray errors after common problems, like a missing #include or a misspelled type name, is minimized. Here is a small example: 1 struct X { 2 unsigned lng x; 3 unsigned int y; 4 }; 5 vodi f(struct X* p) { p.x = 23; } 6 void g(struct X* p) { p->y = p.z; } parse_error.c:2: error: discarding stray 'symbol lng' in declaration specifier parse_error.c:5: error: 'symbol vodi' does not name a type parse_error.c:5: error: request for member 'x' in something not a struct or union, but 'struct X*' parse_error.c:6: error: request for member 'z' in something not a struct or union, but 'struct X*' parse_error.c:6: error: 'struct X*' has no member named 'z' For comparison, these are the error messages by GCC 4.2.1: parse_error.c:2: error: expected ':', ',', ';', '}' or '__attribute__' before 'x' parse_error.c:5: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'f' parse_error.c: In function 'g': parse_error.c:6: error: 'struct X' has no member named 'y' parse_error.c:6: error: request for member 'z' in something not a structure or union cparser can detect a large number of undesirable constructs and warns about them. Here are a few examples: * unused and never read variables 1 #include 2 void f(void) 3 { 4 int x; 5 if ((x = rand())) 6 x = x + 1; 7 } write_only.c:4: warning: variable 'x' is never read 'x' is warned about, because it is only read to calculate its own new value. GCC does not detect this. Similar cases to line 5 have been found by Coverity in the FreeBSD source code recently[5]. * unreachable code 1 #include 2 #include 3 int main(void) 4 { 5 for (int i = 0; i != 10; ++i) { 6 if (rand() == 23) /* mind the missing {} */ 7 puts("ouch"); 8 return 1; 9 } 10 return 0; 11 } unreachable.c:5: warning: step of for-statement is unreachable GCC does not warn here. * format string checker for char and wide char (wchar_t) functions 1 #include 2 #include 3 int main(int argc, char* argv[]) 4 { 5 printf("%u %s", argc); 6 wprintf(L"%S", "hello", 42); 7 return 0; 8 } format.c:5: warning: argument type 'int' does not match conversion specifier '%u' at position 1 format.c:5: warning: too few arguments for format string format.c:6: warning: argument type 'char*' does not match conversion specifier '%S' at position 1 format.c:6: warning: 2 arguments but only 1 format specifier GCC is only capable of checking char, but not wchar_t, format strings. * missing return statments 1 int f(int x) 2 { 3 switch (x) { 4 case 42: return 0; 5 case 23: break; 6 default: return 1; 7 } 8 } missing_return.c:5: warning: control reaches end of non-void function The location is exactly pinpointed: The last statement before leaving the function is the break. In contrast GCC refers to line 8. * typical security warnings 1 #include 2 #include 3 void f(char* x) 4 { 5 if (getpid || &*&x || x == "bla") {} 6 } security.c:5: warning: the address of 'getpid' will always evaluate as 'true' security.c:5: warning: the address of 'x' will always evaluate as 'true' security.c:5: warning: comparison with string literal results in unspecified behaviour * missing and redundant declarations 1 static void f(void); 2 int x; 3 static void f(void) {} decl.c:2: warning: no previous declaration for 'int x' decl.c:1: warning: unnecessary static forward declaration for 'void f(void)' The warning about 'x' is a typical indicator for a missing header include or that the variable should be static. The first declaration of 'f' is unnecessary, because there is no use before the definition of the function. The generated assembler is heavily annotated. Some example C source code: 1 #include 2 #include 3 int f(void) 4 { 5 int x = rand(); 6 if (rand()) 7 puts("bla"); 8 return x; 9 } In the resulting assembler output below you see: * list of predecessors and estimated execution frequency for every basic block /* .L897: preds: 1034, freq: 0.500000 */ * comments telling the correspoding node in the intermediate representation and the original source line (if available) of an instruction. The numbers in [] are internal node counters, which can be referred to in yComp or be used in conjunction with the debugging features of libFIRM. /* ia32_CallT[1086:53] annotate.c:6 */ * function begin/end markers (good for scripting) # -- Begin f # -- End f The resulting assembler output: .section .text .Ltext0: # -- Begin f .p2align 4,,15 .globl f .type f, @function f: /* .L1034: preds: none, freq: 1.000000 */ subl $4, %esp /* be_IncSPIu[1001:12] */ call rand /* ia32_CallT[1059:26] ann.c:5 */ movl %eax, (%esp) /* ia32_StoreM[1213:171] */ call rand /* ia32_CallT[1086:53] ann.c:6 */ testl %eax, %eax /* ia32_TestIu[1220:178] ann.c:6 */ jne .L776 /* ProjX[805:139] ann.c:6 */ /* fallthrough to .L897 */ /* ProjX[806:140] ann.c:6 */ /* .L897: preds: 1034, freq: 0.500000 */ popl %eax /* ia32_PopT[1218:176] */ ret /* be_ReturnX[1031:125] ann.c:8 */ .p2align 4,,7 .L776: /* preds: 1034, freq: 0.500000 */ pushl $Lstr.168 /* ia32_PushT[1223:181] ann.c:7 */ call puts /* ia32_CallT[1118:85] ann.c:7 */ addl $4, %esp /* be_IncSPIu[982:113] */ popl %eax /* ia32_PopT[1221:179] */ ret /* be_ReturnX[1020:107] ann.c:8 */ .size f, .-f # -- End f .section .rodata .type Lstr.168, @object .size Lstr.168, 4 Lstr.168: .string "bla" ---- [1] http://www.libfirm.org/ [2] http://www.info.uni-karlsruhe.de/ [3] http://www.info.uni-karlsruhe.de/software.php/id=6&lang=en [4] specifically ioquake3 [5] for example see http://lists.freebsd.org/pipermail/svn-src-all/2008-November/001623.html From patfbsd at davenulle.org Thu Nov 27 13:10:11 2008 From: patfbsd at davenulle.org (Patrick =?ISO-8859-15?Q?Lamaizi=E8re?=) Date: Thu Nov 27 13:10:19 2008 Subject: Hardware support for AMD Geode CS5536 audio? In-Reply-To: <20713056.post@talk.nabble.com> References: <20080121170155.GC51116@hamlet.SetFilePointer.com> <20713056.post@talk.nabble.com> Message-ID: <20081127221007.31c07ed8@baby-jane> Le Wed, 26 Nov 2008 19:30:41 -0800 (PST), ancelgray a ?crit : Hi, > To AMD CS5536 users: > > This is Andrew Gray. I have finished the audio driver for the AMD > CS5536 companion > chip. It is working on a PC Engines Alix 1C low power board under > FreeBSD 7.0. > It can be found at: > > http://modelofreality.org/snd_amd5536.html You cannot include a file under the GPL license without being contaminated. Even a simple .h with only some #define. I suggest you to use the #define from the auglx OpenBSD's driver, it's free and better documented. http://www.openbsd.org/cgi-bin/cvsweb/~checkout~/src/sys/arch/i386/pci/auglx.c I don't have the hardware to test. My 0,02 euros. Regards. From rea-fbsd at codelabs.ru Thu Nov 27 13:14:47 2008 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Thu Nov 27 13:14:54 2008 Subject: New C compiler and analyzer lang/cparser in ports In-Reply-To: <492F0591.7050807@gmx.de> References: <492F0591.7050807@gmx.de> Message-ID: Christoph, good day. Thu, Nov 27, 2008 at 09:39:45PM +0100, Christoph Mallon wrote: > A few days ago libFIRM[1] and cparser were added to the ports tree. If > you want to see, what other compilers besides GCC have to offer, this > might be of interest for you. libFIRM is a modern optimizing > intermediate representation (IR) library. cparser is a C compiler > providing many useful warnings and uses libFIRM for optimization and > code generation. [...] The whole description looks like that of LLVM GCC port, http://llvm.org/ Do you know something about that project and if yes, could you, please, provide brief comparison of these two? Thanks! -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081127/9d483c73/attachment.pgp From max at love2party.net Thu Nov 27 13:57:04 2008 From: max at love2party.net (Max Laier) Date: Thu Nov 27 13:57:11 2008 Subject: New C compiler and analyzer lang/cparser in ports In-Reply-To: <492F0591.7050807@gmx.de> References: <492F0591.7050807@gmx.de> Message-ID: <200811272257.01287.max@love2party.net> On Thursday 27 November 2008 21:39:45 Christoph Mallon wrote: > A few days ago libFIRM[1] and cparser were added to the ports tree. If > you want to see, what other compilers besides GCC have to offer, this > might be of interest for you. libFIRM is a modern optimizing > intermediate representation (IR) library. cparser is a C compiler > providing many useful warnings and uses libFIRM for optimization and > code generation. Nice stuff! I remember from a practical work with libFIRM that it is quite pleasant to work with (though there is a bit of a learning curve). How are the chances of this code staying under GPL2? I suppose there is no chance to get it dual-licensed under BSDL as well? In addition, how much work is it to build missing backends? It seems like there is no amd64 or sparc support at the moment? -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From christoph.mallon at gmx.de Thu Nov 27 14:22:25 2008 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Thu Nov 27 14:22:33 2008 Subject: New C compiler and analyzer lang/cparser in ports In-Reply-To: References: <492F0591.7050807@gmx.de> Message-ID: <492F1D9E.9040607@gmx.de> Eygene Ryabinkin schrieb: > Christoph, good day. > > Thu, Nov 27, 2008 at 09:39:45PM +0100, Christoph Mallon wrote: >> A few days ago libFIRM[1] and cparser were added to the ports tree. If >> you want to see, what other compilers besides GCC have to offer, this >> might be of interest for you. libFIRM is a modern optimizing >> intermediate representation (IR) library. cparser is a C compiler >> providing many useful warnings and uses libFIRM for optimization and >> code generation. > [...] > > The whole description looks like that of LLVM GCC port, http://llvm.org/ > Do you know something about that project and if yes, could you, please, > provide brief comparison of these two? Both LLVM and FIRM use SSA as an important aspect of their IR. FIRM uses it more extensively in backend. The whole FIRM backend operates on program graphs in SSA form. Spilling and register allocation are decoupled: First spilling is done and after that registers are allocated. There are several spilling algorithms available. The best so far is one based on the idea of Belady's algorithm and it produces very good results. Our novel approach to register allocation and copy coalescing was developed as PhD thesis by Sebastian hack. LLVM uses a more classical approach. It has a combined register allocation and spilling phase using a modified linear scan algorithm. LLVM not only needs an instruction schedule, but also a block schedule. FIRM does block scheduling after register allocation is done. The SSA based spilling of FIRM leads to better code (i.e. less spill and reloads) in most cases. A paper by Matthias Braun and Sebastian Hack submitted to the next CC elaborates on this subject. In the middle end both systems are quite similar, but there are some notable differences. LLVM uses instructions scheduled in basic blocks whereas FIRM resembles the "sea of nodes" proposed by Click and has no schedule in the "middleend". Actually a schedule is introduced after instruction selection. This is necessary, so spilling and register allocation can operator. To be more precise our representation is called an "explicit dependency graph": All necessary dependencies (we use dependencies, not (data) flow) are modelled as edges in the program graph. We use no temporary variables, but every node in the graph represents a value. If a node produces several results, projection nodes are used to get the different aspects. Memory is modelled as SSA value, too. You can have a look at this in our online demo of the compiler. The link is on our website. You can also read a bit more about FIRM there, too. You mentioned LLVM GCC. FIRM only has a stale GCC frontend, which is not maintained anymore. Currently there are three frontends available: C and Java using the Edison Design Group frontend and our own C frontend called "cparser". The latter is probably the more interesting option for you. Did you find the frontend examples interesting or are you mainly interested in optimization and backend aspects? Regards Christoph From rea-fbsd at codelabs.ru Thu Nov 27 14:39:06 2008 From: rea-fbsd at codelabs.ru (Eygene Ryabinkin) Date: Thu Nov 27 14:39:12 2008 Subject: New C compiler and analyzer lang/cparser in ports In-Reply-To: <492F1D9E.9040607@gmx.de> References: <492F0591.7050807@gmx.de> <492F1D9E.9040607@gmx.de> Message-ID: Christoph, Thu, Nov 27, 2008 at 11:22:22PM +0100, Christoph Mallon wrote: > Both LLVM and FIRM use SSA as an important aspect of their IR. FIRM uses [...] Very interesting, thanks for a good review! > Did you find the frontend examples interesting or are you mainly > interested in optimization and backend aspects? Both ;)) Frontends for code generation and optimization/backends for playing with their guts and learning interesting things. Thanks again! -- Eygene _ ___ _.--. # \`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard / ' ` , __.--' # to read the on-line manual )/' _/ \ `-_, / # while single-stepping the kernel. `-'" `"\_ ,_.-;_.-\_ ', fsc/as # _.-'_./ {_.' ; / # -- FreeBSD Developers handbook {_.-``-' {_/ # -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081127/68121630/attachment.pgp From christoph.mallon at gmx.de Thu Nov 27 14:48:56 2008 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Thu Nov 27 14:49:04 2008 Subject: New C compiler and analyzer lang/cparser in ports In-Reply-To: <200811272257.01287.max@love2party.net> References: <492F0591.7050807@gmx.de> <200811272257.01287.max@love2party.net> Message-ID: <492F23D5.3000009@gmx.de> Max Laier schrieb: > On Thursday 27 November 2008 21:39:45 Christoph Mallon wrote: >> A few days ago libFIRM[1] and cparser were added to the ports tree. If >> you want to see, what other compilers besides GCC have to offer, this >> might be of interest for you. libFIRM is a modern optimizing >> intermediate representation (IR) library. cparser is a C compiler >> providing many useful warnings and uses libFIRM for optimization and >> code generation. > > Nice stuff! I remember from a practical work with libFIRM that it is quite > pleasant to work with (though there is a bit of a learning curve). Yes, some design desicions of FIRM seem odd at first - like the use of data dependencies instead of data flow, but once you "get it" it's quite easy. Also you constantly have to maintain SSA form. Sometimes I seems to be a hassle, but usually it prevents you from doing stupid things and when you think about how to do the same thing in non-SSA, you realise how much more hassle that would be. (-: > How are the chances of this code staying under GPL2? I suppose there is no > chance to get it dual-licensed under BSDL as well? I do not want to speculate about this. libFIRM is an university project and I have no idea how hard something like relicensing would be. > In addition, how much work is it to build missing backends? It seems like > there is no amd64 or sparc support at the moment? AMD64 should not be too hard. It is (naturally) very similar to IA32, so basically some extensions to the instruction selection and the ABI transformation should suffice. Other backends need some more work, but a simple instruction selector can be done quite fast. Of course, efficiently using all features of the target machine is considerably more work. If the target is sufficiently "RISCy", all the later phases like spilling and register allocation can directly be used. There are architecures, which are very fancy and need much custom code, but none of the usual architectures like ARM and MIPS fall into this category. There are unfinished backends for other architecures, but the major obstacle to support more backends is the lack of manpower. If you know somebody, who wants to work as Hiwi or who is interested in a study or diploma thesis, send him to us. (-: Regards Christoph From kevin at your.org Thu Nov 27 16:37:17 2008 From: kevin at your.org (Kevin Day) Date: Thu Nov 27 16:37:33 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <7d6fde3d0811271230p512d1325i591e9711caa432d8@mail.gmail.com> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> <86myfl15cl.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> <86wsepymw0.fsf@ds4.des.no> <7d6fde3d0811271230p512d1325i591e9711caa432d8@mail.gmail.com> Message-ID: On Nov 27, 2008, at 2:30 PM, Garrett Cooper wrote: > On Thu, Nov 27, 2008 at 12:03 PM, Kevin Day wrote: >> Just in case anyone needs a real step-by-step guide to getting a >> diskless >> PXE/NFS boot going, I wrote this up a little while ago. >> >> http://sigsegv.org/wiki/How_to_do_a_PXE_netboot_install_of_FreeBSD >> >> The existing documentation was either a bit out of date, glossed >> over some >> of it or just flat out didn't work for me. Takes an existing >> running system, >> clones it into /nfsroot, enables dhcpd/tftpd/nfsd, and relies on >> rc.diskless >> to make the magic happen after boot. > > That's a good brief guide, but a lot of the information is already > contained in: >. > > That reminds me that I need to add some documentation for booting > OpenFirmware Mac's with Netboot (some experiences that I had at Cisco > that I wanted to pass down for others). Saying this carefully so I don't sound like I'm volunteering to update a bunch of docs... :) The problems I encountered with that PXE guide is: * It's now 5 floppies instead of 2 to get sysinstall going. The PXE loader can't handle this (keeps prompting you for the next floppy). Alfred's changes to loader.rc don't work anymore. * The loader has problems with launching gzipped kernels brought over PXE for some reason. I dunno when this started. * vnconfig is now mdconfig * His guide was just to get sysinstall running, not end up with a usable diskless workstation. * It doesn't mention the dhcp option "root-path", which was where I kept getting stuck. It worked flawlessly when 4.0 was -CURRENT, but it's a bit out of date in a few places. From des at des.no Fri Nov 28 01:08:23 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri Nov 28 01:08:30 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <7d6fde3d0811271230p512d1325i591e9711caa432d8@mail.gmail.com> (Garrett Cooper's message of "Thu, 27 Nov 2008 12:30:04 -0800") References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEBF@polaris.maxiscale.com> <86myfl15cl.fsf@ds4.des.no> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEC4@polaris.maxiscale.com> <86wsepymw0.fsf@ds4.des.no> <7d6fde3d0811271230p512d1325i591e9711caa432d8@mail.gmail.com> Message-ID: <86skpcz08q.fsf@ds4.des.no> "Garrett Cooper" writes: > That's a good brief guide, but a lot of the information is already > contained in: > . That's five years out of date, and even five years ago it was a ridiculously complicated way of doing what is essentially a very simple operation. DES -- Dag-Erling Sm?rgrav - des@des.no From des at des.no Fri Nov 28 01:43:40 2008 From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Fri Nov 28 01:43:47 2008 Subject: Hardware support for AMD Geode CS5536 audio? In-Reply-To: <20081127221007.31c07ed8@baby-jane> ("Patrick =?utf-8?Q?Lamai?= =?utf-8?Q?zi=C3=A8re=22's?= message of "Thu, 27 Nov 2008 22:10:07 +0100") References: <20080121170155.GC51116@hamlet.SetFilePointer.com> <20713056.post@talk.nabble.com> <20081127221007.31c07ed8@baby-jane> Message-ID: <86hc5syylw.fsf@ds4.des.no> Patrick Lamaizi?re writes: > You cannot include a file under the GPL license without being > contaminated. Even a simple .h with only some #define. On the contrary, as was settled in SCO v. IBM, constant and structure definitions are not copyrightable if they are dictated by external factors, such as compatibility with existing software or hardware. There are even cases in which actual code is not copyrightable, specifically if the algorithm it implements is so simple that no reasonable programmer would have implemented it in any other way. (SCO tried to argue, among other things, that Linux had stolen errno.h from SCO Unix) DES -- Dag-Erling Sm?rgrav - des@des.no From unixmania at gmail.com Fri Nov 28 03:18:35 2008 From: unixmania at gmail.com (Carlos A. M. dos Santos) Date: Fri Nov 28 03:18:42 2008 Subject: Hardware support for AMD Geode CS5536 audio? In-Reply-To: <86hc5syylw.fsf@ds4.des.no> References: <20080121170155.GC51116@hamlet.SetFilePointer.com> <20713056.post@talk.nabble.com> <20081127221007.31c07ed8@baby-jane> <86hc5syylw.fsf@ds4.des.no> Message-ID: On Fri, Nov 28, 2008 at 7:43 AM, Dag-Erling Sm?rgrav wrote: > Patrick Lamaizi?re writes: >> You cannot include a file under the GPL license without being >> contaminated. Even a simple .h with only some #define. This is not correct, as DES explained. Anyway, the amd5536.h file has a much more serious problem: it contains a license statement but not the name of the copyright owner, which is mandatory. Anonymous contents can not be licensed. > On the contrary, as was settled in SCO v. IBM, constant and structure > definitions are not copyrightable if they are dictated by external > factors, such as compatibility with existing software or hardware. > > There are even cases in which actual code is not copyrightable, > specifically if the algorithm it implements is so simple that no > reasonable programmer would have implemented it in any other way. > > (SCO tried to argue, among other things, that Linux had stolen errno.h > from SCO Unix) > > DES > -- > Dag-Erling Sm?rgrav - des@des.no I mostly agree with you, but considering my previous experiences with GPL-ed header files and their copyright owners(1), I would prefer not to take the risk. Reference 1. http://cvsweb.xfree86.org/cvsweb/xc/lib/GLw/README.txt?rev=1.2 -- cd /usr/ports/sysutils/life make clean From christoph.mallon at gmx.de Fri Nov 28 03:20:11 2008 From: christoph.mallon at gmx.de (Christoph Mallon) Date: Fri Nov 28 03:20:19 2008 Subject: New C compiler and analyzer lang/cparser in ports In-Reply-To: <20081128093858.57826oi96gmzliww@webmail.leidinger.net> References: <492F0591.7050807@gmx.de> <20081128093858.57826oi96gmzliww@webmail.leidinger.net> Message-ID: <492FD3E7.6000205@gmx.de> Alexander Leidinger schrieb: > Quoting Christoph Mallon (from Thu, 27 Nov > 2008 21:39:45 +0100): > >> cparser is a C compiler, which can parse C89 and C99 as well as many >> GCC and some MSVC extensions. The handled GCC extensions include >> __attribute__, inline assembler, computed goto and statement >> expressions. The compiler driver is compatible with with GCC (-fxxx, >> -Wxxx, -M, ...). It also provides many useful analyses for warnings - >> for some examples see below. > > How much of the GCC stuff in /usr/include/cdefs.h would work with > cparser? Info: this is the major part one has to do to make another > compiler ready to compile our kernel. I guess, you mean /usr/include/sys/cdefs.h. Let's have a look. #define __GNUCLIKE_ASM 3 #define __GNUCLIKE_MATH_BUILTIN_CONSTANTS cparser supports GCC style inline assembler (except for x87 floating point constraints, which are not implemented, yet) and also supports builtins like __builtin_nanf. #define __GNUCLIKE___TYPEOF 1 #define __GNUCLIKE___OFFSETOF 1 cparser supports __typeof__ and __builtin_offsetof #define __GNUCLIKE___SECTION 1 This is missing. #define __GNUCLIKE_ATTRIBUTE_MODE_DI 1 The __attribute__((mode($FOO)) insanity is supported. # define __GNUCLIKE_CTOR_SECTION_HANDLING 1 __attribute__((constructor)) and destructor are supported. #define __GNUCLIKE_BUILTIN_CONSTANT_P 1 # define __GNUCLIKE_BUILTIN_VARARGS 1 # define __GNUCLIKE_BUILTIN_STDARG 1 # define __GNUCLIKE_BUILTIN_VAALIST 1 cparser handles these GCC builtins. #define __CC_SUPPORTS_INLINE 1 #define __CC_SUPPORTS___INLINE 1 #define __CC_SUPPORTS___INLINE__ 1 I think we support all alternative spellings with any number of underscores for all keywords. (: I'll skip some simple stuff now, which also works. #define __dead2 __attribute__((__noreturn__)) #define __pure2 __attribute__((__const__)) #define __unused __attribute__((__unused__)) #define __used __attribute__((__used__)) #define __packed __attribute__((__packed__)) #define __aligned(x) __attribute__((__aligned__(x))) #define __section(x) __attribute__((__section__(x))) All of these are parsed and except for __section__ are handled, e.g. __noreturn__ is used for optimization. We also have -Wmissing-noreturn, which warns about functions, which should have this attribute. I'm skipping more attribute stuff. #define __weak_reference(sym,alias) \ __asm__(".weak " #alias); \ __asm__(".equ " #alias ", " #sym) global asm statements are supported, too. The compiler driver is compatible with GCC, but we need to support more switches. This is a matter of time and digging through GCC documentation. Regards Christoph From unixmania at gmail.com Fri Nov 28 03:33:44 2008 From: unixmania at gmail.com (Carlos A. M. dos Santos) Date: Fri Nov 28 03:33:52 2008 Subject: Hardware support for AMD Geode CS5536 audio? In-Reply-To: <20713056.post@talk.nabble.com> References: <20080121170155.GC51116@hamlet.SetFilePointer.com> <20713056.post@talk.nabble.com> Message-ID: On Thu, Nov 27, 2008 at 1:30 AM, ancelgray wrote: > > To AMD CS5536 users: > > This is Andrew Gray. I have finished the audio driver for the AMD CS5536 > companion > chip. It is working on a PC Engines Alix 1C low power board under FreeBSD > 7.0. > It can be found at: > > http://modelofreality.org/snd_amd5536.html > > Let me know how it goes. > > Andrew Gray > > ancelgray AT y a h o o DOT com Andrew, Thanks for the contribution. I have a spare HP thin client with AMD Geode in which I can give this driver a try, if I find some time available next week. I'm a bit concerned about the copyrights, however, as outlined below. The amd5536.c informs that it is "derived from Bruce R. Montegue's AMD CS5530 audio driver and the Linux CS5535 ALSA driver". I did not find the original source files, but supposing that they are licensed under GPL you will need a written permission from the owners to redistribute the code under a different licensing terms. From a strictly legal point of view, even paraphrasing a code may be considered copy. The amd5536.h contains a license statement but not the name of the licensor (copyright owner), which is mandatory. Anonymous contents can not be licensed. -- cd /usr/ports/sysutils/life make clean From Alexander at Leidinger.net Fri Nov 28 00:39:16 2008 From: Alexander at Leidinger.net (Alexander Leidinger) Date: Fri Nov 28 04:24:13 2008 Subject: New C compiler and analyzer lang/cparser in ports In-Reply-To: <492F0591.7050807@gmx.de> References: <492F0591.7050807@gmx.de> Message-ID: <20081128093858.57826oi96gmzliww@webmail.leidinger.net> Quoting Christoph Mallon (from Thu, 27 Nov 2008 21:39:45 +0100): > cparser is a C compiler, which can parse C89 and C99 as well as many > GCC and some MSVC extensions. The handled GCC extensions include > __attribute__, inline assembler, computed goto and statement > expressions. The compiler driver is compatible with with GCC (-fxxx, > -Wxxx, -M, ...). It also provides many useful analyses for warnings > - for some examples see below. How much of the GCC stuff in /usr/include/cdefs.h would work with cparser? Info: this is the major part one has to do to make another compiler ready to compile our kernel. Bye, Alexander. -- What this country needs is a good five-cent nickel. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From avg at icyb.net.ua Fri Nov 28 05:12:40 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Nov 28 05:12:48 2008 Subject: ukbd attachment and root mount In-Reply-To: <492E9F45.2090404@icyb.net.ua> References: <4911BA93.9030006@icyb.net.ua> <491ABFCD.3060309@icyb.net.ua> <491AC502.9000507@icyb.net.ua> <20081112121410.GA24629@icarus.home.lan> <491ACA19.2040008@icyb.net.ua> <20081112123315.GA24907@icarus.home.lan> <491AD0CB.8050309@icyb.net.ua> <20081112132124.GA25637@icarus.home.lan> <491ADB3B.2090000@icyb.net.ua> <492E952A.80104@icyb.net.ua> <492E9F45.2090404@icyb.net.ua> Message-ID: <492FEE43.5060703@icyb.net.ua> on 27/11/2008 15:23 Andriy Gapon said the following: > I increased debug level in uhub and also switched mouse and keyboard > ports hoping that order might matter. It didn't. > > Here's fresh usbdevs output snippet: > Controller /dev/usb2: > addr 1: full speed, self powered, config 1, UHCI root hub(0x0000), > Intel(0x0000), rev 1.00 > uhub2 > port 1 addr 3: low speed, power 100 mA, config 1, USB Keyboard(0x0101), > CHESEN(0x0a81), rev 1.10 > ukbd0 > uhid0 > port 2 addr 2: low speed, power 98 mA, config 1, USB-PS/2 Optical > Mouse(0xc040), Logitech(0x046d), rev 24.30 > ums0 > > And here's a new snippet from cold explore dmesg: > uhub2: uhub_explore: port 1 status 0x0100 0x0001 > + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + So, hm, it looks like a change in connection status is reported but > current status is reported as not connected. > + I wonder why? For now I am blaming this on the keyboard. My wild un-educated guess is that it takes it too long to come back after controller reset. I don't have any other explanation at the moment. I'll try to get another keyboard (from different vendor) and play with it. -- Andriy Gapon From avg at icyb.net.ua Fri Nov 28 05:28:41 2008 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Nov 28 05:28:47 2008 Subject: dd if=/dev/mem can hang a machine? Message-ID: <492FF203.5060405@icyb.net.ua> I have a new machine with DG33TL mainboard (ICH9/G33). In a course of some hacking I ran dd if=/dev/mem ... to scan all memory, this caused the machine to hang. I tried to reproduce and this is 100% reproducible. I am not used to such behavior. In older days I could scan all the memory without any issues. Could this be related to some modern form of memory-mapped IO? Or to Intel Management Engine (that seems t bite into DRAM)? Or something else? Just wondering. -- Andriy Gapon From Alexander at Leidinger.net Fri Nov 28 06:45:31 2008 From: Alexander at Leidinger.net (Alexander Leidinger) Date: Fri Nov 28 09:34:24 2008 Subject: HOWTO in wiki: adding custom dtrace probes in the kernel Message-ID: <20081128154514.82247fe47bn83lkw@webmail.leidinger.net> Hi, while working on adding dtrace probes to the linuxulator, I thought it may be interesting for some people to know how to do this, so I added some text/examples into the wiki how to add static dtrace probes in the kernel: http://wiki.freebsd.org/HOWTO-dtrace-sdt If you are interested in the great debugging possibilities of dtrace, don't be afraid to add some probes to the kernel. It's cheap and easy. Bye, Alexander. -- Leela: You buy one pound of underwear and you're on their list forever. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From fernando.apesteguia at gmail.com Fri Nov 28 09:38:29 2008 From: fernando.apesteguia at gmail.com (=?ISO-8859-1?Q?Fernando_Apestegu=EDa?=) Date: Fri Nov 28 09:38:36 2008 Subject: Fwd: FBSD 7.1 BETA2 and RTL8168/8111 problem In-Reply-To: <1bd550a00811280033x4803b05cyf83446ae15f47086@mail.gmail.com> References: <1bd550a00811280033x4803b05cyf83446ae15f47086@mail.gmail.com> Message-ID: <1bd550a00811280914p19e35e15xf0130d652406bf89@mail.gmail.com> Hi, I forward my orginal mail to freebsd-questions. I hope someone can shed some light here... ---------- Forwarded message ---------- From: Fernando Apestegu?a Date: Fri, Nov 28, 2008 at 9:33 AM Subject: FBSD 7.1 BETA2 and RTL8168/8111 problem To: FreeBSD Hi all, I recently installed FreeBSD 7.1 BETA2 on my system. During installation using FTP option, I could notice the following: After some random time (two, five or six minutes, for instance) the installation stalled and sysinstall lost the connection. I was taken back to the "Select FTP site" screen, and I had to configure my NIC again. Then, the install resumed and I got a usable system (MINIMAL + docs + manpages) But the problem persists. From time to time, it seems the kernel can't see the NIC. It happens during normal operation and no message is shown in /var/log/messages. ifconfig doesn't show my 're0' device, so I can't run dhclient on it and I have to reboot. I have to say that sometimes, even when I reboot, the NIC (RTL8168/8111 PCI Express) is not present. This device works fine in the same computer with either Vista or Fedora 9, though I have to say I had similar problems with earlier versions of Fedora (device disappearing or not present after boot), but after a kernel upgrade everything run smoothly. Does FreeBSD use the same driver than Linux does? Anybody else with this problem? Thanks in advance. From ken at mthelicon.com Fri Nov 28 11:22:56 2008 From: ken at mthelicon.com (Pegasus Mc Cleaft) Date: Fri Nov 28 11:23:03 2008 Subject: Fwd: FBSD 7.1 BETA2 and RTL8168/8111 problem In-Reply-To: <1bd550a00811280914p19e35e15xf0130d652406bf89@mail.gmail.com> References: <1bd550a00811280033x4803b05cyf83446ae15f47086@mail.gmail.com> <1bd550a00811280914p19e35e15xf0130d652406bf89@mail.gmail.com> Message-ID: <200811281922.53179.ken@mthelicon.com> On Friday 28 November 2008 17:14:33 Fernando Apestegu?a wrote: > Hi, > > I forward my orginal mail to freebsd-questions. I hope someone can > shed some light here... > > > ---------- Forwarded message ---------- > From: Fernando Apestegu?a > Date: Fri, Nov 28, 2008 at 9:33 AM > Subject: FBSD 7.1 BETA2 and RTL8168/8111 problem > To: FreeBSD > > > Hi all, > > I recently installed FreeBSD 7.1 BETA2 on my system. During > installation using FTP option, I could notice the following: > > After some random time (two, five or six minutes, for instance) the > installation stalled and sysinstall lost the connection. I was taken > back to the "Select FTP site" screen, and I had to configure my NIC > again. Then, the install resumed and I got a usable system (MINIMAL + > docs + manpages) > > But the problem persists. From time to time, it seems the kernel can't > see the NIC. It happens during normal operation and no message is > shown in /var/log/messages. ifconfig doesn't show my 're0' device, so > I can't run dhclient on it and I have to reboot. I have to say that > sometimes, even when I reboot, the NIC (RTL8168/8111 PCI Express) is > not present. > > This device works fine in the same computer with either Vista or > Fedora 9, though I have to say I had similar problems with earlier > versions of Fedora (device disappearing or not present after boot), > but after a kernel upgrade everything run smoothly. > > Does FreeBSD use the same driver than Linux does? Anybody else with > this problem? > Hi Fernando, I have seen this problem on my machine as well with both 7.X and 8.0-current. I am using a RTL8111C chip on a ICH9 GigaByte GA-48X-DS5 motherboard. If the problem you are experiencing is the same as mine, the clue will be in the dmesg. If you take a look, you may see an error message saying something about a MAC without a PHY (I cant remember the exact message). The motherboard I have has 2 ethernet ports on it, and randomly it would be missing one or both of them. I gave up on trying to get mine to work and turned off the ethernet option in BIOS and installed an Intel Pro/1000 based card. I have found the em driver to be rock stable. Peg From rick-freebsd2008 at kiwi-computer.com Fri Nov 28 20:52:25 2008 From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty) Date: Fri Nov 28 20:52:32 2008 Subject: FreeBSD boot menu is missing In-Reply-To: <861vwx4fd5.fsf@ds4.des.no> References: <2ACA3DE8F9758A48B8BE2C7A847F91F240CE6B@polaris.maxiscale.com> <20081126153510.6062cd55@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CE99@polaris.maxiscale.com> <20081126190545.17b79195@bhuda.mired.org> <2ACA3DE8F9758A48B8BE2C7A847F91F240CEB4@polaris.maxiscale.com> <861vwx4fd5.fsf@ds4.des.no> Message-ID: <20081129042543.GA5094@keira.kiwi-computer.com> On Thu, Nov 27, 2008 at 11:45:10AM +0100, Dag-Erling Sm?rgrav wrote: > > Just make sure you leave a few unallocated blocks at the end of the disk > (for gmirror metadata). In most cases, this happens automatically, Well, you only need to leave room for 512 bytes or one hard drive sector, in the case of gmirror. I've never seen a drive that ends exactly on a cylinder boundary; hardware RAID assumes this too. I would like to hear of a drive which is an exact cylinder multiple and how hardware RAID handles those situations. I always just assume there's extra space that's not consumed. -- Rick C. Petty From laladelausanne at gmail.com Sat Nov 29 03:40:53 2008 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Sat Nov 29 03:40:59 2008 Subject: How to build kernel module spread on subdirectories? In-Reply-To: <711D7381-D852-4B6B-991A-84BA6DEFB679@gmail.com> References: <711D7381-D852-4B6B-991A-84BA6DEFB679@gmail.com> Message-ID: <2A1A4C21-8A2D-4151-BCA0-5FAE1D3BBE86@gmail.com> On 25 Nov 2008, at 15:20 , Nikola Kne?evi? wrote: > I tried to move from OBJS into SRCS (main BSDmakefile now has: SRCS+= > $(ELEMENT_SRCS)), by using something like: > # subdir0 > ELEMENT_SRCS__x =\ > subdir1/file0.cc \ > subdir1/file1.cc > > ... > > But this fails during the linking phase, because the linker is > called with subdir1/file0.o, instead of just file0.o. > > To make something clear, I didn't just rewrite the GNUmakefile to > BSDmakefile, I also followed some of the logic used to build kernel > modules. I'm including bsd.kmod.ko, list sources in SRCS, don't have > any explicit rule to build .o out of .cc/.c. There is no all: > target, as well. Hi, since there were no replies, I went into the various .mk's, and I found some inconsistencies when building modules. If you have a file in a different directory, below the directory where you BSDmakefile is, objects won't be linked nor cleaned properly. Default .c rule builds the object file in the .OBJDIR. But, when we have files with absolute paths in SRCS, they get transfered verbatim to OBJS, transforming only the suffix .c -> .o. When we want to build the module, final rule is (at least on amd64): ${FULLPROG}: ${OBJS} ${LD} ... -o ${.TARGET} ${OBJS} which is wrong, because linker gets the absolute paths to .o files, which do not exist at that place. It would be better to use ${OBJS:T} So I propose the attached patch kmod.ko to circumvent this problem. While I'm at it, I've attached another patch for /usr/share/mk/sys.mk, which resolves a problem when one uses g++ during a building of a kernel module. Cheers, Nikola -------------- next part -------------- A non-text attachment was scrubbed... Name: kmod.mk.patch Type: application/octet-stream Size: 575 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081129/b1f5f100/kmod.mk.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: sys.mk.patch Type: application/octet-stream Size: 353 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20081129/b1f5f100/sys.mk.obj From brucem at mail.cruzio.com Sat Nov 29 07:23:52 2008 From: brucem at mail.cruzio.com (Bruce R. Montague) Date: Sat Nov 29 07:23:59 2008 Subject: Hardware support for AMD Geode CS5536 audio? Message-ID: <200811290822.mAT8MFtl000303@mail.cruzio.com> Hi, re Carlos A. M. dos Santos comment: > The amd5536.c informs that it is "derived from Bruce R. Montegue's AMD > CS5530 audio driver and the Linux CS5535 ALSA driver". I did not find > the original source files, but supposing that they are licensed under > GPL you will need a written permission from the owners to redistribute > the code under a different licensing terms. For orignal source: http://63.249.85.132/gx_audio/Makefile http://63.249.85.132/gx_audio/ns_geode.h http://63.249.85.132/gx_audio/ns_geode.c Doc: http://63.249.85.132/gx_audio/index.html I wrote the CS5530 audio driver referenced around 2001, it was always a FreeBSD driver under a modern BSD-license. The driver was never GPL-licensed. The audio driver was developed on FreeBSD during a contract for NatSemi (who owned the Geode at the time, they had just brought Cyrix). The driver was part of an effort to "undo" drivers that had been put into the Geode's "hypervisor" (sort of an extended ACPI that provided device emulators ("device models"); it turned out that was considered reverse engineering hardware and so was a no-no). The driver was done from scratch, from the manuals, with occasional help from Cyrix engineers. As I recall, FreeBSD helped a little with the final debugging of the Geode hardware, booting a picobsd floppy became part of the standard test suite. At the time, I found it convenient to email floppy-sized picobsd systems that demonstrated a hardware bug to remote hardware engineers. -bruce From eitanadlerlist at gmail.com Sat Nov 29 15:06:52 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sat Nov 29 15:06:59 2008 Subject: keeping track of local modifications Message-ID: <4931CB02.9070904@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm starting to work on local modifications to freeBSD and I was thinking of how to keep track of my local modifications. My first thought was to make a svn repo and and keep the freeBSD source as a "vendor branch". Has anyone else done this? What else do you do to keep track of local modifications? Note: I'm tracking -STABLE; not -CURRENT. As an aside can anyone point me to a relatively easy bug/feature that I can work on as a beginner C coder? - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkxywIACgkQtl8kq+nCzNG9+QCdFZMihzFDBHulzTWKFsTI1i5n 7B4AnRoBW4uU0sCRfMTapO17aY5i5sbD =9Y49 -----END PGP SIGNATURE----- From max at love2party.net Sat Nov 29 15:13:57 2008 From: max at love2party.net (Max Laier) Date: Sat Nov 29 15:14:04 2008 Subject: keeping track of local modifications In-Reply-To: <4931CB02.9070904@gmail.com> References: <4931CB02.9070904@gmail.com> Message-ID: <200811300013.54902.max@love2party.net> On Sunday 30 November 2008 00:06:42 Eitan Adler wrote: > I'm starting to work on local modifications to freeBSD and I was > thinking of how to keep track of my local modifications. > My first thought was to make a svn repo and and keep the freeBSD source > as a "vendor branch". Has anyone else done this? What else do you do to > keep track of local modifications? > Note: I'm tracking -STABLE; not -CURRENT. Use svk. There is information about how to set that up on the wiki: http://wiki.freebsd.org/SubversionPrimer > As an aside can anyone point me to a relatively easy bug/feature that I > can work on as a beginner C coder? Check http://www.freebsd.org/projects/ideas/ -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From gabor at FreeBSD.org Sat Nov 29 15:29:52 2008 From: gabor at FreeBSD.org (=?ISO-8859-1?Q?G=E1bor_K=F6vesd=E1n?=) Date: Sat Nov 29 15:29:59 2008 Subject: keeping track of local modifications In-Reply-To: <4931CB02.9070904@gmail.com> References: <4931CB02.9070904@gmail.com> Message-ID: <4931CC8F.40700@FreeBSD.org> Eitan Adler escribi?: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I'm starting to work on local modifications to freeBSD and I was > thinking of how to keep track of my local modifications. > My first thought was to make a svn repo and and keep the freeBSD source > as a "vendor branch". Has anyone else done this? What else do you do to > keep track of local modifications? > Note: I'm tracking -STABLE; not -CURRENT. > I use Mercurial as described here: http://wiki.freebsd.org/LocalMercurial > As an aside can anyone point me to a relatively easy bug/feature that I > can work on as a beginner C coder? > I suggest that you should browse the unassigned problem reports here from the bin category. Regards, -- Gabor Kovesdan EMAIL: gabor@FreeBSD.org WWW: http://www.kovesdan.org From eitanadlerlist at gmail.com Sat Nov 29 19:01:56 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sat Nov 29 19:02:02 2008 Subject: change to make - error when Makefile doesn't exist Message-ID: <49320219.4070804@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I stupidly forgot to make a backup so I can't provide a diff but in src/usr.bin I changed TryReadMakefile("Makefile"); to if (!TryReadMakefile("Makefile")) Fatal("Makefile could not be opened"); That way you could differentiate between the Makefile not existing and the target not existing. gmake provided the following error message: gmake: *** No targets specified and no makefile found. Stop. - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkyAhkACgkQtl8kq+nCzNEhygCeLZAXcYWje2/95sBkTHKhiwbQ +RAAn0jzYxVngGC8NnpSXXpHgIlQnQN+ =6as2 -----END PGP SIGNATURE----- From eitanadlerlist at gmail.com Sat Nov 29 19:08:44 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sat Nov 29 19:08:55 2008 Subject: keeping track of local modifications In-Reply-To: <200811300013.54902.max@love2party.net> References: <4931CB02.9070904@gmail.com> <200811300013.54902.max@love2party.net> Message-ID: <493203B2.4070109@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Max Laier wrote: > > Use svk. There is information about how to set that up on the wiki: > http://wiki.freebsd.org/SubversionPrimer This is what I am doing *waits for sync to complete* > >> As an aside can anyone point me to a relatively easy bug/feature that I >> can work on as a beginner C coder? > > Check http://www.freebsd.org/projects/ideas/ > Most of these involve C coding beyond my skill level. - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkyA7EACgkQtl8kq+nCzNEO1QCggIzcx40f8IpC2IesFoGGKHcw yY0An3ZrxcCK83c1iJEBnFkBRN3Rz1SW =FKce -----END PGP SIGNATURE----- From eitanadlerlist at gmail.com Sat Nov 29 19:47:36 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sat Nov 29 19:47:43 2008 Subject: minor change to src/usr.bin/window/wwend.c Message-ID: <49320CCD.3020900@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Fixes two errors when building with -ansi -Wall - --- wwend.c.backup 2008-11-29 22:41:42.000000000 -0500 +++ wwend.c 2008-11-29 22:46:08.000000000 -0500 @@ -40,12 +40,14 @@ "$FreeBSD: src/usr.bin/window/wwend.c,v 1.4 2001/05/17 09:38:49 obrien Exp $"; #endif /* not lint */ +#include #include #include "ww.h" #include "tt.h" /*ARGSUSED*/ - -wwend(exit) +int +wwend(int exit) { if (tt.tt_checkpoint) { (void) alarm(0); -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkyDM0ACgkQtl8kq+nCzNF7HgCdG5jqCJA8gX8nCIsraqw2zCqV bEEAoJPg7I3iy/Zv8yDjcSe5bniEvoKs =pcgY -----END PGP SIGNATURE----- From eitanadlerlist at gmail.com Sat Nov 29 20:01:06 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sat Nov 29 20:01:12 2008 Subject: change to ee.c Message-ID: <49320FF7.4040901@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I changed this pursuant to a warning I got from gcc. according to the man page "This avoids the race between testing for a file's existence and opening it for use." Could someone on this list please tell me a) If mkstemp is supposed to be used instead b) if not, why not? I tested this change and was able to spell check files (the function which this changes). As an aside I got an unreproducible segfault 11 when I tried to spellcheck an empty file. When I tried again I did not get the same error. I have the ee.core file. - --- ee.c.back 2008-11-29 22:52:28.000000000 -0500 +++ ee.c 2008-11-29 22:52:35.000000000 -0500 @@ -4386,7 +4386,7 @@ return; } (void)sprintf(template, "/tmp/ee.XXXXXXXX"); - - name = mktemp(&template[0]); + name = mkstemp(&template[0]); fd = open(name, O_CREAT | O_EXCL | O_RDWR, 0600); if (fd < 0) { wmove(com_win, 0, 0); - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkyD/cACgkQtl8kq+nCzNHS/QCghMyQDjyG0dxDgoAj39vBKTBj 6WMAn3H1+6qpCOWR3DloKJlkFeGTdZzp =lE8S -----END PGP SIGNATURE----- From delphij at delphij.net Sat Nov 29 20:10:30 2008 From: delphij at delphij.net (Xin LI) Date: Sat Nov 29 20:10:38 2008 Subject: change to ee.c In-Reply-To: <49320FF7.4040901@gmail.com> References: <49320FF7.4040901@gmail.com> Message-ID: <4932122A.8070209@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Eitan, Eitan Adler wrote: > I changed this pursuant to a warning I got from gcc. > according to the man page "This avoids the race between testing for a > file's existence and opening it for use." > > Could someone on this list please tell me > a) If mkstemp is supposed to be used instead > b) if not, why not? > I tested this change and was able to spell check files (the function > which this changes). > > As an aside I got an unreproducible segfault 11 when I tried to > spellcheck an empty file. When I tried again I did not get the same > error. I have the ee.core file. > > --- ee.c.back 2008-11-29 22:52:28.000000000 -0500 > +++ ee.c 2008-11-29 22:52:35.000000000 -0500 > @@ -4386,7 +4386,7 @@ > return; > } > (void)sprintf(template, "/tmp/ee.XXXXXXXX"); > - name = mktemp(&template[0]); > + name = mkstemp(&template[0]); > fd = open(name, O_CREAT | O_EXCL | O_RDWR, 0600); > if (fd < 0) { > wmove(com_win, 0, 0); Tanks for interested in this but I'm afraid that your patch is incorrect. mkstemp returns a file descriptor rather than a string pointer, therefore, the subsequent open() would have undefined behavior. It looks like that we actually want fd = mkstemp() here. Note that we may want to bring vendor fixes before making any changes to reduce duplicated work... Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkyEioACgkQi+vbBBjt66Dg8QCgw5nCU0G1WnHYtVziiNMpawqh iPwAni7zA4yFnX9waNC0Hmj36rX6yrIG =iJ2c -----END PGP SIGNATURE----- From eitanadlerlist at gmail.com Sat Nov 29 20:22:00 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sat Nov 29 20:22:06 2008 Subject: change to ee.c In-Reply-To: <4932122A.8070209@delphij.net> References: <49320FF7.4040901@gmail.com> <4932122A.8070209@delphij.net> Message-ID: <493214DC.2080904@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Xin LI wrote: > Hi, Eitan, > Tanks for interested in this but I'm afraid that your patch is > incorrect. mkstemp returns a file descriptor rather than a string > pointer, therefore, the subsequent open() would have undefined behavior. > It looks like that we actually want fd = mkstemp() here. Thanks. If this is the case how come gcc did not return any warnings? > > Note that we may want to bring vendor fixes before making any changes to > reduce duplicated work... I was not aware that this was a third party program. I'll look around and see if this was fixed. I'm sort of learning C by fixing these minor bugs so any help is appreciated. > > Cheers, - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkyFNwACgkQtl8kq+nCzNGlRgCeJDMfvnR0ZVtz/k/OjDlrKUUP 3xwAniOJpgueqFh3KVKhk7hS9GFE9fqD =gTGT -----END PGP SIGNATURE----- From bruce at cran.org.uk Sun Nov 30 00:43:40 2008 From: bruce at cran.org.uk (Bruce Cran) Date: Sun Nov 30 00:43:46 2008 Subject: change to ee.c In-Reply-To: <493214DC.2080904@gmail.com> References: <49320FF7.4040901@gmail.com> <4932122A.8070209@delphij.net> <493214DC.2080904@gmail.com> Message-ID: <20081130004331.086941f3@tau> On Sat, 29 Nov 2008 23:21:48 -0500 Eitan Adler wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Xin LI wrote: > > Hi, Eitan, > > > Tanks for interested in this but I'm afraid that your patch is > > incorrect. mkstemp returns a file descriptor rather than a string > > pointer, therefore, the subsequent open() would have undefined > > behavior. It looks like that we actually want fd = mkstemp() here. > Thanks. If this is the case how come gcc did not return any warnings? > > > > Note that we may want to bring vendor fixes before making any > > changes to reduce duplicated work... > I was not aware that this was a third party program. I'll look around > and see if this was fixed. > The version of ee in FreeBSD is fairly old: the latest from http://mahon.cwx.net/ is 1.4.6. Even so, the latest version still generates lots of warnings from gcc because the developer used NULL instead of '\0' (i.e the NULL constant instead of the NUL string). The patch at http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/127986 fixes them; I emailed the developer but got no reply. -- Bruce Cran From rdivacky at freebsd.org Sun Nov 30 01:39:41 2008 From: rdivacky at freebsd.org (Roman Divacky) Date: Sun Nov 30 01:39:49 2008 Subject: minor change to src/usr.bin/window/wwend.c In-Reply-To: <49320CCD.3020900@gmail.com> References: <49320CCD.3020900@gmail.com> Message-ID: <20081130093456.GA89886@freebsd.org> On Sat, Nov 29, 2008 at 10:47:25PM -0500, Eitan Adler wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Fixes two errors when building with -ansi -Wall > > > - --- wwend.c.backup 2008-11-29 22:41:42.000000000 -0500 > +++ wwend.c 2008-11-29 22:46:08.000000000 -0500 > @@ -40,12 +40,14 @@ > "$FreeBSD: src/usr.bin/window/wwend.c,v 1.4 2001/05/17 09:38:49 > obrien Exp $"; > #endif /* not lint */ > > +#include > #include > #include "ww.h" > #include "tt.h" > > /*ARGSUSED*/ > - -wwend(exit) > +int > +wwend(int exit) > { > if (tt.tt_checkpoint) { > (void) alarm(0); please use -current, I already fixed this in -current a week or so ago... From eitanadlerlist at gmail.com Sun Nov 30 06:11:25 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sun Nov 30 06:11:31 2008 Subject: minor change to src/usr.bin/window/wwend.c In-Reply-To: <20081130093456.GA89886@freebsd.org> References: <49320CCD.3020900@gmail.com> <20081130093456.GA89886@freebsd.org> Message-ID: <49329F00.2020702@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Roman Divacky wrote: please use -current - src or list? I already fixed this in -current a week or so ago... Sorry - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkynwAACgkQtl8kq+nCzNGgKgCcCR0Bj9LUffhWF9Ah6ccc5OS8 FusAnRDOOdpkz4XKuT6A1QuqtDNhxaWt =D19D -----END PGP SIGNATURE----- From eitanadlerlist at gmail.com Sun Nov 30 06:12:39 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sun Nov 30 06:12:45 2008 Subject: change to ee.c In-Reply-To: <20081130004331.086941f3@tau> References: <49320FF7.4040901@gmail.com> <4932122A.8070209@delphij.net> <493214DC.2080904@gmail.com> <20081130004331.086941f3@tau> Message-ID: <49329F4A.3070004@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bruce Cran wrote: > The version of ee in FreeBSD is fairly old: the latest from > http://mahon.cwx.net/ is 1.4.6. How difficult would it be to bring it up to date? How come it has not been updated so far? - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkyn0oACgkQtl8kq+nCzNFxbwCfRdNcyTOBWCsPHVCXTi57EN/N ve0An2+XR4l/KWgC2XZC4Lf65K6d2+RW =jUlC -----END PGP SIGNATURE----- From gary.jennejohn at freenet.de Sun Nov 30 07:24:43 2008 From: gary.jennejohn at freenet.de (Gary Jennejohn) Date: Sun Nov 30 07:24:50 2008 Subject: dd if=/dev/mem can hang a machine? In-Reply-To: <492FF203.5060405@icyb.net.ua> References: <492FF203.5060405@icyb.net.ua> Message-ID: <20081130162437.1bae4371@ernst.jennejohn.org> On Fri, 28 Nov 2008 15:28:35 +0200 Andriy Gapon wrote: > > I have a new machine with DG33TL mainboard (ICH9/G33). > In a course of some hacking I ran dd if=/dev/mem ... to scan all memory, > this caused the machine to hang. > I tried to reproduce and this is 100% reproducible. > > I am not used to such behavior. In older days I could scan all the > memory without any issues. > > Could this be related to some modern form of memory-mapped IO? Or to > Intel Management Engine (that seems t bite into DRAM)? > Or something else? > > Just wondering. > That's what I would assume. With some hardware just reading a register can be harmful. --- Gary Jennejohn From hartmut.brandt at dlr.de Sun Nov 30 07:44:03 2008 From: hartmut.brandt at dlr.de (Hartmut Brandt) Date: Sun Nov 30 07:44:10 2008 Subject: change to make - error when Makefile doesn't exist In-Reply-To: <49320219.4070804@gmail.com> References: <49320219.4070804@gmail.com> Message-ID: <4932B193.7030106@dlr.de> Eitan Adler wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I stupidly forgot to make a backup so I can't provide a diff but in > src/usr.bin > I changed > TryReadMakefile("Makefile"); > to > if (!TryReadMakefile("Makefile")) > Fatal("Makefile could not be opened"); > That way you could differentiate between the Makefile not existing and > the target not existing. > > gmake provided the following error message: > gmake: *** No targets specified and no makefile found. Stop. > > It is perfectly valid to have no makefile yet to run make. Make has a number of builtin rules. If, for example, you have a file x.c then 'make x' or 'make x.o' make will cause make to compile x.c even without a makefile. But if you have no Makefile, you must specify a target. Otherwise how would make know what to make? So the error message is quite exact: it is an error to have no makefile AND to specify no target. harti From eitanadlerlist at gmail.com Sun Nov 30 07:53:39 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sun Nov 30 07:53:45 2008 Subject: change to make - error when Makefile doesn't exist In-Reply-To: <4932B193.7030106@dlr.de> References: <49320219.4070804@gmail.com> <4932B193.7030106@dlr.de> Message-ID: <4932B6F7.4020206@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hartmut Brandt wrote: > It is perfectly valid to have no makefile yet to run make. Make has a > number of builtin rules. If, for example, you have a file x.c then 'make > x' or 'make x.o' make will cause make to compile x.c even without a > makefile. But if you have no Makefile, you must specify a target. > Otherwise how would make know what to make? So the error message is > quite exact: it is an error to have no makefile AND to specify no target. > Ah - I was not aware of this - thanks. - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkytvcACgkQtl8kq+nCzNFRTwCfcI3cfiDSC3sprwqOBtf3Xu/r Cy8AnRpNDqoZW1RVD+++12dmp8EqT8X2 =UUnY -----END PGP SIGNATURE----- From max at love2party.net Sun Nov 30 08:44:19 2008 From: max at love2party.net (Max Laier) Date: Sun Nov 30 08:44:26 2008 Subject: keeping track of local modifications In-Reply-To: <493203B2.4070109@gmail.com> References: <4931CB02.9070904@gmail.com> <200811300013.54902.max@love2party.net> <493203B2.4070109@gmail.com> Message-ID: <200811301744.16947.max@love2party.net> On Sunday 30 November 2008 04:08:34 Eitan Adler wrote: ... > >> As an aside can anyone point me to a relatively easy bug/feature that I > >> can work on as a beginner C coder? > > > > Check http://www.freebsd.org/projects/ideas/ > > Most of these involve C coding beyond my skill level. Find a project you care about - no matter how difficult it might seem - and start reading code & hacking. If you are truly passionate about your project you can learn quite a bit in short time. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From fbsd.hackers at rachie.is-a-geek.net Sun Nov 30 10:02:13 2008 From: fbsd.hackers at rachie.is-a-geek.net (Mel) Date: Sun Nov 30 10:02:20 2008 Subject: How to build kernel module spread on subdirectories? In-Reply-To: <2A1A4C21-8A2D-4151-BCA0-5FAE1D3BBE86@gmail.com> References: <711D7381-D852-4B6B-991A-84BA6DEFB679@gmail.com> <2A1A4C21-8A2D-4151-BCA0-5FAE1D3BBE86@gmail.com> Message-ID: <200811301843.28564.fbsd.hackers@rachie.is-a-geek.net> On Saturday 29 November 2008 12:40:47 Nikola Kne?evi? wrote: > On 25 Nov 2008, at 15:20 , Nikola Kne?evi? wrote: > > I tried to move from OBJS into SRCS (main BSDmakefile now has: SRCS+= > > $(ELEMENT_SRCS)), by using something like: > > # subdir0 > > ELEMENT_SRCS__x =\ > > subdir1/file0.cc \ > > subdir1/file1.cc > > > > ... > > > > But this fails during the linking phase, because the linker is > > called with subdir1/file0.o, instead of just file0.o. > > > > To make something clear, I didn't just rewrite the GNUmakefile to > > BSDmakefile, I also followed some of the logic used to build kernel > > modules. I'm including bsd.kmod.ko, list sources in SRCS, don't have > > any explicit rule to build .o out of .cc/.c. There is no all: > > target, as well. > > Hi, > > since there were no replies, I went into the various .mk's, and I > found some inconsistencies when building modules. If you have a file > in a different directory, below the directory where you BSDmakefile > is, objects won't be linked nor cleaned properly. The base of the FreeBSD build system, is that SRCS contains file names. Not pathnames. Use .PATH directive if sources are elsewhere. This is the only thing you cannot easily change and should not globally change. A simple example of this is sbin/fsdb/Makefile which uses sources from sbin/fsck_ffs. If you really need it, you should override compilation rules in your own BSDmakefile, like so: .c.o: ${CC} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET} .cpp.o: ${CXX} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET} As a general rule, a directory should contain "one project", use bsd.subdir.mk to descend. Absolute paths in a Makefile only work on YOUR machine, so don't do it. -- Mel From bruce at cran.org.uk Sun Nov 30 10:32:08 2008 From: bruce at cran.org.uk (Bruce Cran) Date: Sun Nov 30 10:32:14 2008 Subject: change to ee.c In-Reply-To: <49329F4A.3070004@gmail.com> References: <49320FF7.4040901@gmail.com> <4932122A.8070209@delphij.net> <493214DC.2080904@gmail.com> <20081130004331.086941f3@tau> <49329F4A.3070004@gmail.com> Message-ID: <20081130103156.6e4da70c@tau> On Sun, 30 Nov 2008 09:12:26 -0500 Eitan Adler wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Bruce Cran wrote: > > > The version of ee in FreeBSD is fairly old: the latest from > > http://mahon.cwx.net/ is 1.4.6. > How difficult would it be to bring it up to date? How come it has not > been updated so far? I'd guess it hasn't been updated because it works as it is and nobody has taken an interest in bringing it up to date. I don't know how many patches have been applied locally to it but that would be the main challenge in importing a newer version: you'd have to go through the FreeBSD CVS history and check that any changes/fixes that were made to FreeBSD's version are either already included in the new release or ensure that those changes get re-applied to the new version. -- Bruce Cran From kientzle at freebsd.org Sun Nov 30 11:26:13 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Nov 30 11:26:20 2008 Subject: keeping track of local modifications In-Reply-To: <4931CB02.9070904@gmail.com> References: <4931CB02.9070904@gmail.com> Message-ID: <4932E8CF.9040501@freebsd.org> Eitan Adler wrote: > As an aside can anyone point me to a relatively easy bug/feature that I > can work on as a beginner C coder? There are thousands of such; could you narrow it down a little bit? Are you interested in kernel hacking? Device support? Core libraries? Networking? Utilities? Porting? There are a couple of idea pages floating around and many discussions in public mailing lists, blogs, and other places: * Search the source code for "TODO" or "XXX" to find comments about things that could use cleaning or improvement. As a bonus, these usually comment things that someone thinks should be done, so you might have a little easier job selling your solution. * Search the web for "FreeBSD GSoC" or "FreeBSD Summer of Code" to find ideas that are intended to be 3-month projects targeted at student-level developers. You'll find links both to past projects (a fair number of which are still not yet entirely completed; maybe you could help?) and web pages and mailing list discussions about possible future projects. * Search the web for "Junior Kernel Hacker" for other ideas that people have come up with over the years. * Go to FreeBSD.org and skim the mailing list archives to find ideas and see what problems people are having. hackers@ and current@ are good starting points, some of the more specialized lists can also be interesting reading. You might find problems that you can solve yourself or you might find an ongoing project that could use a little help. Projects like FreeBSD work best when there are several people working on any given area. Of course, how a lot of people get involved is simply to install and use FreeBSD for a little while, and find something that doesn't seem quite right: * a hardware device on your system that's not fully supported? * a utility that doesn't work the way you think it should? * an application that doesn't run on FreeBSD as well as it does on some other platform? In my case, I'd used FreeBSD for many years, decided I didn't like the installer (still don't, by the way) but found that writing a new installer was too big a project for the limited time I had available. So I cast around and found my niche working on archiving tools. (Maybe Ivan Voras is interested in having help with his finstall project?) As you figure it out, it's usually a good idea to ask on mailing lists (hackers@ is good) or IRC to see if other people are encountering the same problem or if someone is already working on something. That kind of discussion can help you get more complete background on a particular problem, including approaches that other people have tried or even partial code for fixes that were never completed. (There's a lot of interesting bits sitting around people's hard drives that are worth the effort to study, test, and fix.) Perhaps most importantly, by talking about your work-in-progress, you have a better chance of connecting with a committer who will help get your work into the tree. A lot of excellent ideas never make it into FreeBSD because the author never talked to anyone until they were "done" and the result couldn't really fit into FreeBSD correctly without a lot more work. On this latter point, it can help to read carefully through old commit logs, study past work in that area, and ask questions specifically of developers who have done work in that area. (Though it's usually better to ask first in a public forum like hackers@ or current@; individual developers are sometimes very busy or on vacation or just slow to respond for various reasons.) Most importantly, have fun and remember that most of us are volunteers who enjoy using and working on FreeBSD in our (often quite limited) spare time. On the one hand, that sometimes makes us slow to answer: If I only have a couple of hours a week, I'd usually rather spend it coding than typing long answers to questions that people could answer themselves. On the other hand, it also means I enjoy talking about FreeBSD and sometimes get carried away writing overly-long email epics with lots of detail about stuff that noone really cares about. Welcome to the zoo! Tim P.S. libarchive has a number of mid-sized projects that I'd like to see done, including improvements to the ISO reader (tar's ability to read ISOs is a popular feature that merits some additional work), RMT support for tar and cpio, a pax front-end, an ISO writer, and a bunch of work to identify common functions in tar and cpio and find clean ways to move them into libarchive so they can be effectively shared. This last in particular doesn't require complex coding since the core functionality is already written, but does require a lot of care with library API design (a subject about which I have strong opinions) and refactoring existing code. From kientzle at freebsd.org Sun Nov 30 11:41:54 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Nov 30 11:42:01 2008 Subject: change to ee.c In-Reply-To: <493214DC.2080904@gmail.com> References: <49320FF7.4040901@gmail.com> <4932122A.8070209@delphij.net> <493214DC.2080904@gmail.com> Message-ID: <4932EC6A.1070205@freebsd.org> Eitan Adler wrote: > Xin LI wrote: >>Tanks for interested in this but I'm afraid that your patch is >>incorrect. mkstemp returns a file descriptor rather than a string >>pointer, therefore, the subsequent open() would have undefined behavior. >> It looks like that we actually want fd = mkstemp() here. > > Thanks. If this is the case how come gcc did not return any warnings? Try to think of compiler warnings as a luxury. For a variety of reasons (some having to do with the somewhat permissive nature of the C language), it's extremely hard for the compiler to detect anything other than the most blatant screw-ups. Whenever using a new function, pay very careful attention to the man pages: man mkstemp clearly says that mkstemp returns a file descriptor: "The mkstemp() function ... creates the template file, mode 0600, returning a file descriptor opened for reading and writing." Tim From jhs at berklix.org Sun Nov 30 12:31:45 2008 From: jhs at berklix.org (Julian Stacey) Date: Sun Nov 30 12:31:54 2008 Subject: dd if=/dev/mem can hang a machine? In-Reply-To: Your message "Sun, 30 Nov 2008 16:24:37 +0100." <20081130162437.1bae4371@ernst.jennejohn.org> Message-ID: <200811302014.mAUKE5YF046736@fire.js.berklix.net> Gary Jennejohn wrote: > On Fri, 28 Nov 2008 15:28:35 +0200 > Andriy Gapon wrote: > > > > > I have a new machine with DG33TL mainboard (ICH9/G33). > > In a course of some hacking I ran dd if=/dev/mem ... to scan all memory, > > this caused the machine to hang. > > I tried to reproduce and this is 100% reproducible. > > > > I am not used to such behavior. In older days I could scan all the > > memory without any issues. > > > > Could this be related to some modern form of memory-mapped IO? Or to > > Intel Management Engine (that seems t bite into DRAM)? > > Or something else? > > > > Just wondering. > > > > That's what I would assume. With some hardware just reading a register > can be harmful. I just crashed 3 normally stable machines trying that. I only tried for casual interest. I acknowledge Gary's comment above :-) dd if=/dev/mem of=/dev/null FreeBSD lapa.js.berklix.net 4.11-RELEASE FreeBSD 4.11-RELEASE #0: Fri Jul 7 17:56:30 CEST 2006 jhs@lapa.jhs.private:/usr/src/sys/compile/LAPA.small i386 FreeBSD laps.js.berklix.net 7.0-RELEASE FreeBSD 7.0-RELEASE #2: Mon Sep 8 15:39:53 CEST 2008 jhs@laps.js.berklix.net:/usr1/ftp/pri/FreeBSD/releases/7.0-RELEASE/src/sys/i386/compile/LAPS.small i386 FreeBSD john.js.berklix.net 7.1-BETA2 FreeBSD 7.1-BETA2 #0: Sun Oct 12 20:59:28 UTC 2008 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Cheers, Julian -- Julian Stacey: BSDUnixLinux C Prog Admin SysEng Consult Munich www.berklix.com Mail plain ASCII text. HTML & Base64 text are spam. www.asciiribbon.org From laladelausanne at gmail.com Sun Nov 30 14:57:11 2008 From: laladelausanne at gmail.com (=?UTF-8?Q?Nikola_Kne=C5=BEevi=C4=87?=) Date: Sun Nov 30 14:57:18 2008 Subject: How to build kernel module spread on subdirectories? In-Reply-To: <200811301843.28564.fbsd.hackers@rachie.is-a-geek.net> References: <711D7381-D852-4B6B-991A-84BA6DEFB679@gmail.com> <2A1A4C21-8A2D-4151-BCA0-5FAE1D3BBE86@gmail.com> <200811301843.28564.fbsd.hackers@rachie.is-a-geek.net> Message-ID: <94D09AB0-86B6-4D91-BD61-AB02A12CC260@gmail.com> On 30 Nov 2008, at 18:43 , Mel wrote: >> since there were no replies, I went into the various .mk's, and I >> found some inconsistencies when building modules. If you have a file >> in a different directory, below the directory where you BSDmakefile >> is, objects won't be linked nor cleaned properly. > > The base of the FreeBSD build system, is that SRCS contains file > names. Not > pathnames. Use .PATH directive if sources are elsewhere. This is the > only Hi Mel, thanks for the clarification. I didn't know that SRCS has to contain only filenames. > thing you cannot easily change and should not globally change. A > simple > example of this is sbin/fsdb/Makefile which uses sources from sbin/ > fsck_ffs. > > If you really need it, you should override compilation rules in your > own > BSDmakefile, like so: > .c.o: > ${CC} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET} > > .cpp.o: > ${CXX} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET} > So, I'm now using only filenames in SRCS and the .PATH target. As I described in my initial mail, I have to use two makefiles. One is the main Makefile, while the other has to be generated. In the generated .mk, I add to the SRCS, and I create the .PATH target with other (necessary) directories. Unfortunately, make depend doesn't work as it should be. This is the important snippet of my BSDmakefile: ---8<--- elements.mk: elements.conf $(CLICKDIR)/click-buildtool elem2make --bsd elements.conf > elements.mk .MAKEFILEDEPS: elements.mk .sinclude "elements.mk" .include --->8--- When I run make depend, it only includes SRCSs from BSDmakefile, not those from elements.mk. Is there a way to overcome this? I can manually run make elements.mk, but it is a bit tedious. Cheers, Nikola From simias.n at gmail.com Sun Nov 30 14:57:40 2008 From: simias.n at gmail.com (Lionel Flandrin) Date: Sun Nov 30 14:57:48 2008 Subject: Hardware support for AMD Geode CS5536 audio? In-Reply-To: <20713056.post@talk.nabble.com> References: <20080121170155.GC51116@hamlet.SetFilePointer.com> <20713056.post@talk.nabble.com> Message-ID: <20081130232851.GA1214@casibsd.svkt.org> On Wed, Nov 26, 2008 at 07:30:41PM -0800, ancelgray wrote: > > To AMD CS5536 users: > > This is Andrew Gray. I have finished the audio driver for the AMD CS5536 > companion > chip. It is working on a PC Engines Alix 1C low power board under FreeBSD > 7.0. > It can be found at: > > http://modelofreality.org/snd_amd5536.html > > Let me know how it goes. I own a fitpc[1] that runs an AMD Geode CPU with the AMD CS5536 chip. I followed the README (kldloaded the module, ran the two sysctl) and everything seemed to work fine: - As soon as I loaded the module I got (in the system messages): pcm0: port 0xd400-0xd47f irq 10 at device 15.3 on pci0 pcm0: [ITHREAD] pcm0: - cat /dev/sndstat says: FreeBSD Audio Driver (newpcm: 32bit 2007061600/i386) Installed devices: pcm0: at io 0xd400 irq 10 kld snd_cs5536 [MPSAFE] \ (2p:0v/1r:0v channels duplex default) However if I try to play something with mpg123 I get no sound and "pcm0: bm0 bus master error" every half second or so in the system messages. Please tell me if there's anything else I can provide to help you. [1] http://www.fit-pc.com/new/fit-pc-1-0-specifications.html Cheers, -- Lionel Flandrin From kientzle at freebsd.org Sun Nov 30 16:22:59 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Nov 30 16:23:05 2008 Subject: How to build kernel module spread on subdirectories? In-Reply-To: <94D09AB0-86B6-4D91-BD61-AB02A12CC260@gmail.com> References: <711D7381-D852-4B6B-991A-84BA6DEFB679@gmail.com> <2A1A4C21-8A2D-4151-BCA0-5FAE1D3BBE86@gmail.com> <200811301843.28564.fbsd.hackers@rachie.is-a-geek.net> <94D09AB0-86B6-4D91-BD61-AB02A12CC260@gmail.com> Message-ID: <49332E5C.9020303@freebsd.org> Nikola Kne?evi? wrote: > As I described in my initial mail, I have to use two makefiles. One is > the main Makefile, while the other has to be generated. In the > generated .mk, I add to the SRCS, and I create the .PATH target with > other (necessary) directories. Unfortunately, make depend doesn't work > as it should be. > .MAKEFILEDEPS: elements.mk > > .sinclude "elements.mk" > .include > --->8--- > > When I run make depend, it only includes SRCSs from BSDmakefile, not > those from elements.mk. I would try adding a "beforedepend" requirement: beforedepend: elements.mk Look at /usr/share/mk/bsd.dep.mk, which has the 'make depend' logic. It supports optional "beforedepend" and "afterdepend" targets. Tim From bsd.quest at googlemail.com Sun Nov 30 17:39:34 2008 From: bsd.quest at googlemail.com (Alexej Sokolov) Date: Sun Nov 30 17:39:41 2008 Subject: remapping kernel buffer in VMS of user process Message-ID: <20081201013851.GA20549@debian.samsung.router> Hello, I would like to remap some buffers allocated in kernel space to memory space of certain process. For Example, in attach function of driver: static int driver_attach { ... struct vmspace *vms = some_thread->td_proc->p_vmspace; bufp = malloc (PAGE_SIZE, M_DEVBUF, M_NOWAIT); /* How to create in vms of some_thread->td_proc remapping of buffer pointed (in kernel) by bufp ? some_thread should access the buffer using its virtual user addresses and driver should access the same data using its kernel virtual addresses (bufp) */ ... } -- Alexej Sokolov From eitanadlerlist at gmail.com Sun Nov 30 18:22:40 2008 From: eitanadlerlist at gmail.com (Eitan Adler) Date: Sun Nov 30 18:22:48 2008 Subject: keeping track of local modifications In-Reply-To: <4932E8CF.9040501@freebsd.org> References: <4931CB02.9070904@gmail.com> <4932E8CF.9040501@freebsd.org> Message-ID: <49334A62.7000800@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Kientzle wrote: > Eitan Adler wrote: > There are thousands of such; could you narrow it down > a little bit? Are you interested in kernel hacking? > Device support? Core libraries? Networking? Utilities? > Porting? Utilities mostly for now. > > * Search the source code for "TODO" or "XXX" to find > comments about things that could use cleaning or > improvement. As a bonus, these usually comment things > that someone thinks should be done, so you might have > a little easier job selling your solution. *slaps self for not thinking of this* Thanks for reminding me of this. I'm now working on * TODO: Make this better, so that "./a//b/./c/" == "a/b/c" part of src/usr.bin/tar/util.c I found a copy of abspath() in usr.sbin/pkg_install/info/perform.c This is the exact function needed to fix the TODO. Where should I move the function to so that they could share it? Should I just copy the function (for some reason I doubt it)? realpath(3) is in sys/param.h and stdlib.h Should I move the function there? > * Search the web for "FreeBSD GSoC" or "FreeBSD Summer of Code" > to find ideas that are intended to be 3-month projects > targeted at student-level developers. Interesting idea - thanks > Perhaps most importantly, by talking about your > work-in-progress, you have a better chance of connecting > with a committer who will help get your work into the > tree. A lot of excellent ideas never make it into > FreeBSD because the author never talked to anyone > until they were "done" and the result couldn't really fit > into FreeBSD correctly without a lot more work. I will do this > On the other hand, it also means I enjoy talking about FreeBSD > and sometimes get carried away writing overly-long email > epics with lots of detail about stuff that noone really > cares about. I really hope you don't think I'm no one. > > Welcome to the zoo! Thanks? > > Tim > > P.S. libarchive has a number of mid-sized projects... I actually picked this project after greping for TODO but before reading this. Interesting. - -- Eitan Adler GNU Key fingerptrint: 2E13 BC16 5F54 0FBD 62ED 42B6 B65F 24AB E9C2 CCD1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEUEARECAAYFAkkzSmIACgkQtl8kq+nCzNFeYQCdGydbomyScD3HCFPq9Y6MgGsw GLYAlRTb0Dv0IDHbHOivsmNdD0kyBiA= =WUos -----END PGP SIGNATURE----- From gnemmi at gmail.com Sun Nov 30 18:59:21 2008 From: gnemmi at gmail.com (Gonzalo Nemmi) Date: Sun Nov 30 18:59:28 2008 Subject: keeping track of local modifications In-Reply-To: <4932E8CF.9040501@freebsd.org> References: <4931CB02.9070904@gmail.com> <4932E8CF.9040501@freebsd.org> Message-ID: <200812010038.16064.gnemmi@gmail.com> On Sunday 30 November 2008 5:26:07 pm Tim Kientzle wrote: > Eitan Adler wrote: > > As an aside can anyone point me to a relatively easy bug/feature that I > > can work on as a beginner C coder? > > There are thousands of such; could you narrow it down > a little bit? Are you interested in kernel hacking? > Device support? Core libraries? Networking? Utilities? > Porting? > > There are a couple of idea pages floating around and > many discussions in public mailing lists, blogs, and other > places: > > * Search the source code for "TODO" or "XXX" to find > comments about things that could use cleaning or > improvement. As a bonus, these usually comment things > that someone thinks should be done, so you might have > a little easier job selling your solution. > > * Search the web for "FreeBSD GSoC" or "FreeBSD Summer of Code" > to find ideas that are intended to be 3-month projects > targeted at student-level developers. You'll find links > both to past projects (a fair number of which are still > not yet entirely completed; maybe you could help?) and > web pages and mailing list discussions about possible > future projects. > > * Search the web for "Junior Kernel Hacker" for other > ideas that people have come up with over the years. > > * Go to FreeBSD.org and skim the mailing list archives > to find ideas and see what problems people are having. > hackers@ and current@ are good starting points, some > of the more specialized lists can also be interesting > reading. You might find problems that you can solve > yourself or you might find an ongoing project that > could use a little help. Projects like FreeBSD work best > when there are several people working on any given area. > > Of course, how a lot of people get involved is simply > to install and use FreeBSD for a little while, and find > something that doesn't seem quite right: > * a hardware device on your system that's not fully supported? > * a utility that doesn't work the way you think it should? > * an application that doesn't run on FreeBSD as well as it > does on some other platform? > > In my case, I'd used FreeBSD for many years, decided I didn't > like the installer (still don't, by the way) but found that > writing a new installer was too big a project for the limited > time I had available. So I cast around and found my niche > working on archiving tools. (Maybe Ivan Voras is interested > in having help with his finstall project?) > > As you figure it out, it's usually a good idea to ask > on mailing lists (hackers@ is good) or IRC to see if > other people are encountering the same problem or if > someone is already working on something. That kind of > discussion can help you get more complete background > on a particular problem, including approaches that other > people have tried or even partial code for fixes that > were never completed. (There's a lot of interesting > bits sitting around people's hard drives that are > worth the effort to study, test, and fix.) > > Perhaps most importantly, by talking about your > work-in-progress, you have a better chance of connecting > with a committer who will help get your work into the > tree. A lot of excellent ideas never make it into > FreeBSD because the author never talked to anyone > until they were "done" and the result couldn't really fit > into FreeBSD correctly without a lot more work. > > On this latter point, it can help to read carefully > through old commit logs, study past work in that area, > and ask questions specifically of developers who have > done work in that area. (Though it's usually better to > ask first in a public forum like hackers@ or current@; > individual developers are sometimes very busy or on > vacation or just slow to respond for various reasons.) > > Most importantly, have fun and remember that most of us are > volunteers who enjoy using and working on FreeBSD in our (often > quite limited) spare time. On the one hand, that sometimes > makes us slow to answer: If I only have a couple of hours > a week, I'd usually rather spend it coding than typing > long answers to questions that people could answer themselves. > On the other hand, it also means I enjoy talking about FreeBSD > and sometimes get carried away writing overly-long email > epics with lots of detail about stuff that noone really > cares about. Eitan cares and I do too ... Thanks a lot for your _really_good_ email :) -- Blessings Gonzalo Nemmi From steve at Watt.COM Sun Nov 30 20:40:35 2008 From: steve at Watt.COM (Steve Watt) Date: Sun Nov 30 20:40:41 2008 Subject: tcsh loses the foreground process group? Message-ID: <20081201042037.GA43208@wattres.Watt.COM> (Please don't cc: me on replies, I can usually keep up with -hackers.) I've run into this about 5 or 6 times recently, and it seems to have survived the last couple of world updates. I'm running 6-STABLE (6.4-PRE as of 24 Nov right now), tcsh 6.15.00, which shows tcsh 6.15.00 (Astron) 2007-03-03 (i386-intel-FreeBSD) options wide,nls,dl,al,kan,sm,rh,color,filec as $version. The symptom is that when I do a long-ish running task inside a `` expansion that I then ^C, nobody gets the foreground process group... I never get a prompt back after the ^C, and ^T gets me load: 0.27 no foreground process group Sending SIGCHLD and/or SIGCONT to the tcsh doesn't seem to make any difference at all. The tcsh is sitting in "pause" WCHAN, which seems sensible. A truss on tcsh reveals (oddly enough) that it's sitting in sigsuspend(). Sending it signals makes it loop through a wait4() call and go back into sigsuspend(). It happens running either as root (from sudo) or as a regular user. It happens under xterm, under ssh sessions, and on a direct console login. One portable reproduction: # cd /usr/src # less `egrep -lir '^Foo.*baz' *` ^Cload: 0.02 no foreground process group (I typed ^C ^T) SIGKILL to the shell seems to be the only way to get things back to normal. I'm not sure how to isolate it further - it seems likely that this is a bug in the kernel, rather than in tcsh, but the session management stuff is pretty hairy in both places. Does anyone have debugging thoughts? -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.5" / 37N 20' 15.3" Internet: steve @ Watt.COM Whois: SW32-ARIN Free time? There's no such thing. It just comes in varying prices... From kientzle at freebsd.org Sun Nov 30 22:19:41 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Nov 30 22:19:47 2008 Subject: keeping track of local modifications In-Reply-To: <49334A62.7000800@gmail.com> References: <4931CB02.9070904@gmail.com> <4932E8CF.9040501@freebsd.org> <49334A62.7000800@gmail.com> Message-ID: <493381F7.4090301@freebsd.org> >>* Search the source code for "TODO" or "XXX" to find >> comments ... > > *slaps self for not thinking of this* > Thanks for reminding me of this. I'm now working on > * TODO: Make this better, so that "./a//b/./c/" == "a/b/c" > part of src/usr.bin/tar/util.c > I found a copy of abspath() in usr.sbin/pkg_install/info/perform.c > This is the exact function needed to fix the TODO. > > Where should I move the function to so that they could share it? Should > I just copy the function (for some reason I doubt it)? > realpath(3) is in sys/param.h and stdlib.h > Should I move the function there? Ah. You picked one of mine. I'll follow up with you off-list. ;-) Tim From perryh at pluto.rain.com Sun Nov 30 22:36:49 2008 From: perryh at pluto.rain.com (perryh@pluto.rain.com) Date: Sun Nov 30 22:36:56 2008 Subject: keeping track of local modifications In-Reply-To: <4932E8CF.9040501@freebsd.org> References: <4931CB02.9070904@gmail.com> <4932E8CF.9040501@freebsd.org> Message-ID: <49337f04.p8QqvfzTga07ypa6%perryh@pluto.rain.com> Tim Kientzle wrote: > ... most of us are volunteers who enjoy using and working on > FreeBSD in our (often quite limited) spare time ... If I only > have a couple of hours a week, I'd usually rather spend it coding > ... Sounds familiar :) Getting back to the OP's original question, and in light of the limited time that many of us have available, I was wondering which of the readily-available VCS would impose the least overhead on someone who has very little experience with any open source VCS (and thus is going to have to learn *something* new). After looking at the pages recommended by others in this thread, I wonder if there are other possibilities which one should consider. * http://wiki.freebsd.org/SubversionPrimer I got the impression that SVN is quite resource-hungry, both in disk space and in bandwidth, and has an extremely steep learning curve. While a committer clearly has to deal with SVN, I was left wondering if it really had much to offer the more casual hacker, esp. one who is not already familiar with it. In particular, given that one will likely have already installed /usr/src/... from the distribution, I was put off by the apparent need to download another entire instance. * http://wiki.freebsd.org/LocalMercurial This seems less of a resource hog, and (if I am understanding matters correctly) is able to start from the installed /usr/src/... rather than requiring the would-be hacker to download a redundant instance, but I was concerned that the page may not be up to date with current FreeBSD development methodology (e.g. csup vs cvsup). In case it makes any difference, I've used SCCS and RCS a little (but neither all that recently), and have been using ClearCase a great deal for the last several years (but it is not a candidate for this inquiry since I'm not licensed to use it outside the office). From kientzle at freebsd.org Sun Nov 30 23:07:10 2008 From: kientzle at freebsd.org (Tim Kientzle) Date: Sun Nov 30 23:07:28 2008 Subject: keeping track of local modifications In-Reply-To: <49337f04.p8QqvfzTga07ypa6%perryh@pluto.rain.com> References: <4931CB02.9070904@gmail.com> <4932E8CF.9040501@freebsd.org> <49337f04.p8QqvfzTga07ypa6%perryh@pluto.rain.com> Message-ID: <49338D15.7080803@freebsd.org> perryh@pluto.rain.com wrote: > Tim Kientzle wrote: > >>... most of us are volunteers who enjoy using and working on >>FreeBSD in our (often quite limited) spare time ... If I only >>have a couple of hours a week, I'd usually rather spend it coding >>... > > Sounds familiar :) > > Getting back to the OP's original question, and in light of the > limited time that many of us have available, I was wondering which of > the readily-available VCS would impose the least overhead on someone > who has very little experience with any open source VCS (and thus is > going to have to learn *something* new). No matter what, I would plan on devoting at least a long weekend to learning a new VCS and getting a local mirror set up. In terms of learning new skills, I don't think you'll go far wrong with any of SVN, Hg, or git. For my part, I actually like SVN quite a lot. In part because it's close enough to CVS (which is in turn close to RCS) for folks familiar with that model to transition pretty easily. I've never used Mercurial or Git, so can't say anything for sure about those. It sounds like it's a little trickier to set up a local SVN mirror than one of the others. In terms of disk and bandwidth, I think SVNs requirements have been overstated just a bit. Unless you have pretty tight disk constraints or a pretty slow connection, I don't think you'll really notice the difference. Tim From max at love2party.net Sun Nov 30 23:30:21 2008 From: max at love2party.net (Max Laier) Date: Sun Nov 30 23:30:27 2008 Subject: keeping track of local modifications In-Reply-To: <49337f04.p8QqvfzTga07ypa6%perryh@pluto.rain.com> References: <4931CB02.9070904@gmail.com> <4932E8CF.9040501@freebsd.org> <49337f04.p8QqvfzTga07ypa6%perryh@pluto.rain.com> Message-ID: <200812010830.17259.max@love2party.net> On Monday 01 December 2008 07:07:00 perryh@pluto.rain.com wrote: > Tim Kientzle wrote: > > ... most of us are volunteers who enjoy using and working on > > FreeBSD in our (often quite limited) spare time ... If I only > > have a couple of hours a week, I'd usually rather spend it coding > > ... > > Sounds familiar :) > > Getting back to the OP's original question, and in light of the > limited time that many of us have available, I was wondering which of > the readily-available VCS would impose the least overhead on someone > who has very little experience with any open source VCS (and thus is > going to have to learn *something* new). After looking at the pages > recommended by others in this thread, I wonder if there are other > possibilities which one should consider. > > * http://wiki.freebsd.org/SubversionPrimer > > I got the impression that SVN is quite resource-hungry, both in > disk space and in bandwidth, and has an extremely steep learning > curve. While a committer clearly has to deal with SVN, I was left > wondering if it really had much to offer the more casual hacker, > esp. one who is not already familiar with it. In particular, given > that one will likely have already installed /usr/src/... from the > distribution, I was put off by the apparent need to download > another entire instance. > > * http://wiki.freebsd.org/LocalMercurial > > This seems less of a resource hog, and (if I am understanding > matters correctly) is able to start from the installed /usr/src/... > rather than requiring the would-be hacker to download a redundant > instance, but I was concerned that the page may not be up to date > with current FreeBSD development methodology (e.g. csup vs cvsup). If you want to contribute back, this is *not* the way to go. Patches from anything other than SVN and maybe CVS are mostly useless. The local hg/git approach is nice if you are already familiar with hg or git and just want to keep some patch sets for yourself. If you are looking to keep/develop a patch set and eventually share it with the world, svn or svk is the way to go. Yes, a full svk mirror takes up 3.5g of space, but that's not even two bucks at today's storage costs. The issue with the initial setup is a different thing, but once that is done (at your local university, employer, or the like) svn/svk is really resource efficient. On top of that you will find that "svn ann" is a very powerful tool to figure out why a certain line of code is the way it is (much more pleasant to use than cvsweb, too - esp. when you are on a slow network connection). So it really depends on what your goal is. If you are (as the OP) looking to contribute back to the community, there is really no way around svn - sorry. If you are looking for some means to store your favorite patches from the lists and some of your own, the local hg/git stuff might be a better fit. > In case it makes any difference, I've used SCCS and RCS a little (but > neither all that recently), and have been using ClearCase a great > deal for the last several years (but it is not a candidate for this > inquiry since I'm not licensed to use it outside the office). No idea about ClearCase, but SCCS and RCS are not too far away from SVN ... natural progression. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From Alexander at Leidinger.net Sun Nov 30 23:36:20 2008 From: Alexander at Leidinger.net (Alexander Leidinger) Date: Mon Dec 1 04:25:33 2008 Subject: change to ee.c In-Reply-To: <20081130103156.6e4da70c@tau> References: <49320FF7.4040901@gmail.com> <4932122A.8070209@delphij.net> <493214DC.2080904@gmail.com> <20081130004331.086941f3@tau> <49329F4A.3070004@gmail.com> <20081130103156.6e4da70c@tau> Message-ID: <20081201083604.545172dag60hadc0@webmail.leidinger.net> Quoting Bruce Cran (from Sun, 30 Nov 2008 10:31:56 -0800): > On Sun, 30 Nov 2008 09:12:26 -0500 > Eitan Adler wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Bruce Cran wrote: >> >> > The version of ee in FreeBSD is fairly old: the latest from >> > http://mahon.cwx.net/ is 1.4.6. >> How difficult would it be to bring it up to date? How come it has not >> been updated so far? > > I'd guess it hasn't been updated because it works as it is and nobody > has taken an interest in bringing it up to date. I don't know how many > patches have been applied locally to it but that would be the main > challenge in importing a newer version: you'd have to go through the > FreeBSD CVS history and check that any changes/fixes that were made to > FreeBSD's version are either already included in the new release or > ensure that those changes get re-applied to the new version. It _may_ be more easy to find out which version is in our source tree, and make a diff between the original vendor version and what we have. Depending on the amount of changes there, this is faster than to real all the version control logs to determine if there's a difference or not. In the end you have to read some logs too, but only those, which change lines which a different from those of the vendor version (e.g. via the annotated view at cvsweb.freebsd.org). Bye, Alexander. -- In this vale Of toil and sin Your head grows bald But not your chin. -- Burma Shave http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137