From mwm-keyword-freebsdhackers2.e313df at mired.org  Mon Jun  1 00:15:52 2009
From: mwm-keyword-freebsdhackers2.e313df at mired.org (Mike Meyer)
Date: Mon Jun  1 00:15:59 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <20090531235943.GA77374@logik.internal.network>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531201445.GA82420@logik.internal.network>
	<alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl>
	<0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl>
	<20090531235943.GA77374@logik.internal.network>
Message-ID: <20090531201408.45dd4334@bhuda.mired.org>

On Mon, 1 Jun 2009 00:59:43 +0100
xorquewasp@googlemail.com wrote:

> There is one last thing I'd like clarified. From the zpool
> manpage:
> 
>   In order  to take advantage of these features, a pool must make use of
>   some form of redundancy, using either mirrored or raidz  groups.  While
>   ZFS  supports running in a non-redundant configuration, where each root
>   vdev is simply a disk or file, this is strongly discouraged.  A  single
>   case of bit corruption can render some or all of your data unavailable.
> 
> Is this supposed to mean:
> 
>   "ZFS is more fragile than most. If you don't use redundancy, one
>    case of bit corruption will destroy the filesystem"
> 
> Or:
> 
>   "Hard disks explode often. Use redundancy."

How about (from an old disk recover paper):

    Disks, unlike software, sometimes fail. Using redundancy can help
    you prevent this from resulting in data loss.

That said, there aren't many file systems that can recover from data
errors in the underlying storage. ZFS appropriately configured is one.
I don't believe the default config is appropriate, though. You need
both checksum on and copies > 1 on, and the latter isn't the
default. It's probably better to let zpool provide the redundancy via
a mirror or raid configuration than to let zfs do it anyway.

       <mike
-- 
Mike Meyer <mwm@mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun  1 00:26:36 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun  1 00:26:42 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <20090531201408.45dd4334@bhuda.mired.org>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531201445.GA82420@logik.internal.network>
	<alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl>
	<0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl>
	<20090531235943.GA77374@logik.internal.network>
	<20090531201408.45dd4334@bhuda.mired.org>
Message-ID: <alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>

>    Disks, unlike software, sometimes fail. Using redundancy can help

modern SATA drives fail VERY often. about 30% of drives i bought recently 
failed in less than a year.

> both checksum on and copies > 1 on, and the latter isn't the
> default. It's probably better to let zpool provide the redundancy via
> a mirror or raid configuration than to let zfs do it anyway.

ZFS copies are far from what i consider useful.

for example you set copies=2. You write a file, and get 2 copies.

Then one disk with one copy fails, then you put another, do resilver but 
ZFS DOES NOT rebuild second copy.

You need to write a program that will just rewrite all files to make this.
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun  1 00:30:35 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun  1 00:30:46 2009
Subject: Getting rid of pxeboot loader requests for .gz an .bz2 files
In-Reply-To: <4A230199.6090107@iem.pw.edu.pl>
References: <4A230199.6090107@iem.pw.edu.pl>
Message-ID: <alpine.BSF.2.00.0906010227080.27329@wojtek.tensor.gdynia.pl>

> #ifdef LOADER_BZIP2_SUPPORT
>    &bzipfs_fsops,
> #endif
>
> Is it the correct idea to:
> make -C /usr/src/sys/boot -DLOADER_NO_GZIP_SUPPORT
> -DLOADER_NO_BZIP2_SUPPORT .... ?
>
yes, but then you will not get any compression support, like compressed 
ramdisk image. but it's probably OK for you.
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun  1 00:31:13 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun  1 00:31:19 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <8FFF597348E24C8085BEDE423DC4A6A3@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531163305.6c7d0cd5@bhuda.mired.org>
	<8FFF597348E24C8085BEDE423DC4A6A3@uk.tiscali.intl>
Message-ID: <alpine.BSF.2.00.0906010230460.27329@wojtek.tensor.gdynia.pl>

> Yep it probably isn't clear enough, it does mention stuff about spreading it
> across vdevs, but doesn't say striped.

isn't spreading and stripping actually the same?
From onemda at gmail.com  Mon Jun  1 01:04:57 2009
From: onemda at gmail.com (Paul B. Mahol)
Date: Mon Jun  1 01:05:05 2009
Subject: Getting rid of pxeboot loader requests for .gz an .bz2 files
In-Reply-To: <4A230199.6090107@iem.pw.edu.pl>
References: <4A230199.6090107@iem.pw.edu.pl>
Message-ID: <3a142e750905311743g100eecc8h749d184fc740cf90@mail.gmail.com>

On 5/31/09, Jedrzej Kalinowski <kalinoj1@iem.pw.edu.pl> wrote:
> Hello,
>
> I've got a FreeBSD CURRENT diskless boot set up.
>
> In the logs of tftpd-hpa I can see that every and each file from
> boot.4th to kernel modules is tried to be fetched in compressed format
> (.gz, .bz2) before it is loaded in its pure form by the pxe-enabled loader.
>
> So the sequence is: file.gz->file.bz2->file.
>
> I don't have compressed versions of these files in my environment, so I
> would like to get rid of these requests. Just to make my boot lighter -
> thus faster.
>
> I'm looking for some hints on how to do it in the sources and I only
> found it on lines 88 - 93 of /usr/src/sys/boot/i386/loader/conf.c, namely:
>
> #ifdef LOADER_GZIP_SUPPORT
>     &gzipfs_fsops,
> #endif
> #ifdef LOADER_BZIP2_SUPPORT
>     &bzipfs_fsops,
> #endif
>
> Is it the correct idea to:
> make -C /usr/src/sys/boot -DLOADER_NO_GZIP_SUPPORT
> -DLOADER_NO_BZIP2_SUPPORT .... ?
>
> Or is it the other place I should look for?

/etc/make.conf ?

-- 
Paul
From fjwcash at gmail.com  Mon Jun  1 02:44:29 2009
From: fjwcash at gmail.com (Freddie Cash)
Date: Mon Jun  1 02:44:35 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <20090531235943.GA77374@logik.internal.network>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531201445.GA82420@logik.internal.network>
	<alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl>
	<0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl>
	<20090531235943.GA77374@logik.internal.network>
Message-ID: <b269bc570905311914k5f0aefe2n5c87a881fd81d5fb@mail.gmail.com>

On Sun, May 31, 2009 at 4:59 PM,  <xorquewasp@googlemail.com> wrote:
> There is one last thing I'd like clarified. From the zpool
> manpage:
>
> ?In order ?to take advantage of these features, a pool must make use of
> ?some form of redundancy, using either mirrored or raidz ?groups. ?While
> ?ZFS ?supports running in a non-redundant configuration, where each root
> ?vdev is simply a disk or file, this is strongly discouraged. ?A ?single
> ?case of bit corruption can render some or all of your data unavailable.
>
> Is this supposed to mean:
>
> ?"ZFS is more fragile than most. If you don't use redundancy, one
> ? case of bit corruption will destroy the filesystem"
>
> Or:
>
> ?"Hard disks explode often. Use redundancy."

Unless you specify mirror or raidz on the create/add line, zfs (in
essence) creates a RAID0 stripe of all the vdevs.  Hence, if a single
drive dies, the whole thing dies.  Just like in a normal
hardware/software RAID0 array.  Nothing special or new here.

Just like "normal" RAID, unless you add redundancy (RAID1/5/6) to a
stripe set, losing a single disk means losing the whole array.
-- 
Freddie Cash
fjwcash@gmail.com
From asmodean at list.ru  Mon Jun  1 06:21:13 2009
From: asmodean at list.ru (Asmodean Dark)
Date: Mon Jun  1 06:21:20 2009
Subject: 7.1-STABLE crash
Message-ID: <E1MB0eY-0005z1-00.asmodean-list-ru@f230.mail.ru>

Hello,

We have cluster of FreeBSD VPN servers with running mpd3.18 and sometimes it crash:

# uname -a
FreeBSD vpn 7.1-STABLE FreeBSD 7.1-STABLE #13: Wed Mar 18 14:53:13 YEKT 2009     root@vpn:/usr/src/sys/i386/compile/kconf  i386

# dmesg
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address   = 0x8
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0x806dcb88
stack pointer           = 0x28:0xe70775c4
frame pointer           = 0x28:0xe70775e8
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 22 (em0 taskq)
trap number             = 12

(kgdb) bt
#0  0x8063d6b0 in doadump ()
#1  0x8063dc44 in boot ()
#2  0x8063e0ca in panic ()
#3  0x807dab3d in trap_fatal ()
#4  0x807daeba in trap_pfault ()
#5  0x807db7bd in trap ()
#6  0x807c2a3b in calltrap ()
#7  0x806dcb88 in rn_match ()
#8  0x806ddc8a in rn_lookup ()
#9  0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894
#10 0x80710c3d in ipfw_check_in (arg=0x0, m0=0xe7017700, ifp=0x91c5a800, dir=1, inp=0x0) at ../../../netinet/ip_fw_pfil.c:125
#11 0x806dc20f in pfil_run_hooks ()
#12 0x80713984 in ip_input (m=0x91954c00) at ../../../netinet/ip_input.c:416
#13 0x806ec0d9 in ng_iface_rcvdata ()
#14 0x806e9570 in ng_apply_item ()
#15 0x806e8569 in ng_snd_item ()
#16 0x806e9570 in ng_apply_item ()
#17 0x806e8569 in ng_snd_item ()
#18 0x806e9570 in ng_apply_item ()
#19 0x806e8569 in ng_snd_item ()
#20 0x806f16a7 in ng_ppp_proto_recv ()
#21 0x806f3ed2 in ng_ppp_rcvdata ()
#22 0x806e9570 in ng_apply_item ()
#23 0x806e8569 in ng_snd_item ()
#24 0x806e9570 in ng_apply_item ()
#25 0x806e8569 in ng_snd_item ()
#26 0x806ee3c3 in ng_ksocket_incoming2 ()
#27 0x806e969d in ng_apply_item ()
#28 0x806ea8aa in ngintr ()
#29 0x806dab72 in swi_net ()
#30 0x8061e265 in ithread_loop ()
#31 0x8061adf5 in fork_exit ()
#32 0x807c2ab0 in fork_trampoline ()

What can I do with it? Are additional info needed?

Thanks.
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun  1 09:03:00 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun  1 09:03:07 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <45F774FB6EDB488099ADF76790ABBD65@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531163305.6c7d0cd5@bhuda.mired.org>
	<8FFF597348E24C8085BEDE423DC4A6A3@uk.tiscali.intl>
	<alpine.BSF.2.00.0906010230460.27329@wojtek.tensor.gdynia.pl>
	<45F774FB6EDB488099ADF76790ABBD65@uk.tiscali.intl>
Message-ID: <alpine.BSF.2.00.0906011101130.29890@wojtek.tensor.gdynia.pl>

> In the case of zfs yes, but not always. Eg you could have a concatenated
> volume. Where you only start writing to the second disk when the 1st is
> full.

i don't know how ZFS exactly allocates space, but i use gconcat with UFS 
and it isn't true.

UFS do "jump" between zones (called cyllinder group) when files are 
written to prevent unevenly filling them, so every zone always has some 
space to allocate.

The effect is that both drives gets quite evenly filled
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun  1 09:05:04 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun  1 09:05:12 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <442651299CA2445991E65F64AF9E65D7@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><20090531201445.GA82420@logik.internal.network><alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl><0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl><20090531235943.GA77374@logik.internal.network><20090531201408.45dd4334@bhuda.mired.org>
	<alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>
	<442651299CA2445991E65F64AF9E65D7@uk.tiscali.intl>
Message-ID: <alpine.BSF.2.00.0906011103160.29890@wojtek.tensor.gdynia.pl>

> You shouldn't need to alter the copies attribute to recover from disk
> failures as the normal raid should take care of that. What the copies is

I don't think we understand each other. I say that when i want 2 copies, 
ZFS should rebuild second copy if it's gone and i run resilver.

it does not, what doesn't make sense for me.

From kraduk at googlemail.com  Mon Jun  1 08:32:19 2009
From: kraduk at googlemail.com (krad)
Date: Mon Jun  1 11:24:23 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <20090531235943.GA77374@logik.internal.network>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><20090531201445.GA82420@logik.internal.network><alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl><0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl>
	<20090531235943.GA77374@logik.internal.network>
Message-ID: <3346D963EE2E4D28AE89CB71C2F92939@uk.tiscali.intl>

Zfs has been designed for highly scalable redundant disk pools therefore
using it on a single drive kind of goes against it ethos. Remember a lot of
the blurb in the man page was written by sun and therefore is written with
corporates in mind, therefore the cost with of the data vs an extra drive
being so large why wouldn't you make it redundant.

Having said that sata drives are cheap these days so you would have to be on
the tightest of budgets not to do a mirror.

Having said all this we quite often us zfs on a single drive, well sort of.
The sun clusters have external storage for the shared file systems. These
are usually a bunch of drives, raid 5, 10 or whatever. Then export a single
lun, which is presented to the various nodes. There is a zpool created on
this LUN. So to all intents and purposes zfs thinks its on a single drive
(the redundancy provided by the external array). This is common practice and
we see no issues with it.

-----Original Message-----
From: owner-freebsd-hackers@freebsd.org
[mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of
xorquewasp@googlemail.com
Sent: 01 June 2009 01:00
To: freebsd-hackers@freebsd.org
Subject: Re: Request for opinions - gvinum or ccd?

There is one last thing I'd like clarified. From the zpool
manpage:

  In order  to take advantage of these features, a pool must make use of
  some form of redundancy, using either mirrored or raidz  groups.  While
  ZFS  supports running in a non-redundant configuration, where each root
  vdev is simply a disk or file, this is strongly discouraged.  A  single
  case of bit corruption can render some or all of your data unavailable.

Is this supposed to mean:

  "ZFS is more fragile than most. If you don't use redundancy, one
   case of bit corruption will destroy the filesystem"

Or:

  "Hard disks explode often. Use redundancy."

_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

From kraduk at googlemail.com  Mon Jun  1 08:36:15 2009
From: kraduk at googlemail.com (krad)
Date: Mon Jun  1 11:24:35 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <alpine.BSF.2.00.0906010230460.27329@wojtek.tensor.gdynia.pl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531163305.6c7d0cd5@bhuda.mired.org>
	<8FFF597348E24C8085BEDE423DC4A6A3@uk.tiscali.intl>
	<alpine.BSF.2.00.0906010230460.27329@wojtek.tensor.gdynia.pl>
Message-ID: <45F774FB6EDB488099ADF76790ABBD65@uk.tiscali.intl>

In the case of zfs yes, but not always. Eg you could have a concatenated
volume. Where you only start writing to the second disk when the 1st is
full.

-----Original Message-----
From: Wojciech Puchar [mailto:wojtek@wojtek.tensor.gdynia.pl] 
Sent: 01 June 2009 01:31
To: krad
Cc: 'Mike Meyer'; freebsd-hackers@freebsd.org; 'Mike Meyer';
xorquewasp@googlemail.com
Subject: RE: Request for opinions - gvinum or ccd?

> Yep it probably isn't clear enough, it does mention stuff about spreading
it
> across vdevs, but doesn't say striped.

isn't spreading and stripping actually the same?

From kraduk at googlemail.com  Mon Jun  1 08:42:27 2009
From: kraduk at googlemail.com (krad)
Date: Mon Jun  1 11:24:51 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><20090531201445.GA82420@logik.internal.network><alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl><0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl><20090531235943.GA77374@logik.internal.network><20090531201408.45dd4334@bhuda.mired.org>
	<alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>
Message-ID: <442651299CA2445991E65F64AF9E65D7@uk.tiscali.intl>

You shouldn't need to alter the copies attribute to recover from disk
failures as the normal raid should take care of that. What the copies is
useful for is when you get undetected write errors on the drive due to crc
collisions or the drive simply being rubbish

Zfs will intelligently assign the copies across multiple drives if possible,
so if you had 3 vdevs and copies set to three, One copy should end up on
each vdev. Note this isn't the same as mirroring, as each vdev could be a
raidz2 group. With copies=1 you would get a 3rd of the file on each. With
copies 3 you would get a full version of each.

This is obviously very costly on drive space though. It is tunable per file
system though so you don't have to enable it for the whole pool just the
bits you want.

-----Original Message-----
From: owner-freebsd-hackers@freebsd.org
[mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of Wojciech Puchar
Sent: 01 June 2009 01:26
To: Mike Meyer
Cc: freebsd-hackers@freebsd.org; xorquewasp@googlemail.com
Subject: Re: Request for opinions - gvinum or ccd?

>    Disks, unlike software, sometimes fail. Using redundancy can help

modern SATA drives fail VERY often. about 30% of drives i bought recently 
failed in less than a year.

> both checksum on and copies > 1 on, and the latter isn't the
> default. It's probably better to let zpool provide the redundancy via
> a mirror or raid configuration than to let zfs do it anyway.

ZFS copies are far from what i consider useful.

for example you set copies=2. You write a file, and get 2 copies.

Then one disk with one copy fails, then you put another, do resilver but 
ZFS DOES NOT rebuild second copy.

You need to write a program that will just rewrite all files to make this.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

From tevans.uk at googlemail.com  Mon Jun  1 12:50:16 2009
From: tevans.uk at googlemail.com (Tom Evans)
Date: Mon Jun  1 12:50:22 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <3346D963EE2E4D28AE89CB71C2F92939@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531201445.GA82420@logik.internal.network>
	<alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl>
	<0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl>
	<20090531235943.GA77374@logik.internal.network>
	<3346D963EE2E4D28AE89CB71C2F92939@uk.tiscali.intl>
Message-ID: <1243860612.9871.68.camel@strangepork.london.mintel.ad>

On Mon, 2009-06-01 at 09:32 +0100, krad wrote:
> Zfs has been designed for highly scalable redundant disk pools therefore
> using it on a single drive kind of goes against it ethos. Remember a lot of
> the blurb in the man page was written by sun and therefore is written with
> corporates in mind, therefore the cost with of the data vs an extra drive
> being so large why wouldn't you make it redundant.
> 
> Having said that sata drives are cheap these days so you would have to be on
> the tightest of budgets not to do a mirror.
> 
> Having said all this we quite often us zfs on a single drive, well sort of.
> The sun clusters have external storage for the shared file systems. These
> are usually a bunch of drives, raid 5, 10 or whatever. Then export a single
> lun, which is presented to the various nodes. There is a zpool created on
> this LUN. So to all intents and purposes zfs thinks its on a single drive
> (the redundancy provided by the external array). This is common practice and
> we see no issues with it.

By doing this surely you lose a lot of the self healing that ZFS offers?
For instance, if the underlying vdev is just a raid5, then a disk
failure combined with an undetected checksum error on a different disk
would lead you to lose all your data. Or am I missing something?

(PS, top posting is bad)

Tom

From kraduk at googlemail.com  Mon Jun  1 12:19:57 2009
From: kraduk at googlemail.com (krad)
Date: Mon Jun  1 13:24:06 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <alpine.BSF.2.00.0906011103160.29890@wojtek.tensor.gdynia.pl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><20090531201445.GA82420@logik.internal.network><alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl><0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl><20090531235943.GA77374@logik.internal.network><20090531201408.45dd4334@bhuda.mired.org>
	<alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>
	<442651299CA2445991E65F64AF9E65D7@uk.tiscali.intl>
	<alpine.BSF.2.00.0906011103160.29890@wojtek.tensor.gdynia.pl>
Message-ID: <3ED3189838484C8FBDECE5478A0DDFF4@uk.tiscali.intl>

Its all done on write, so if you update the file it will have multiple
copies again

This explains it quite well

http://blogs.sun.com/relling/entry/zfs_copies_and_data_protection

-----Original Message-----
From: Wojciech Puchar [mailto:wojtek@wojtek.tensor.gdynia.pl] 
Sent: 01 June 2009 10:05
To: krad
Cc: 'Mike Meyer'; freebsd-hackers@freebsd.org; xorquewasp@googlemail.com
Subject: RE: Request for opinions - gvinum or ccd?

> You shouldn't need to alter the copies attribute to recover from disk
> failures as the normal raid should take care of that. What the copies is

I don't think we understand each other. I say that when i want 2 copies, 
ZFS should rebuild second copy if it's gone and i run resilver.

it does not, what doesn't make sense for me.


From kraduk at googlemail.com  Mon Jun  1 13:19:53 2009
From: kraduk at googlemail.com (krad)
Date: Mon Jun  1 13:59:49 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <1243860612.9871.68.camel@strangepork.london.mintel.ad>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531201445.GA82420@logik.internal.network>
	<alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl>
	<0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl>
	<20090531235943.GA77374@logik.internal.network>
	<3346D963EE2E4D28AE89CB71C2F92939@uk.tiscali.intl>
	<1243860612.9871.68.camel@strangepork.london.mintel.ad>
Message-ID: <20754E8E3F5940E0875AD0BC76A78BAB@uk.tiscali.intl>

no you would only loose the data for that block. Zfs also checksums meta
data, but by default keeps multiple copies of it so that's fairly resilient.
If you had the copies set to > 1 then you wouldn't loose the block either,
unless you were real unlucky. 

It's just about pushing the odds back further and further. If you are super
paranoid by all means put in 48 drive, group them into 5 x 8 drive raidz2
vdevs, have a bunch of hot spares, and enable copies=5 for blocks and
metadata, then duplicate the system and put the other box on another
continent and zfs send all you updates every 15 mins via a private
deadicated. This will all prove very resilient, but you will get very little
% storage from your drives, and have quite a large bandwidth bill 8)

Oh and don't forget the scrub you disk regularly. BTW that would rebuild any
missing copies as well (eg if you increase the number of copies after data
is stored on the fs)










-----Original Message-----
From: Tom Evans [mailto:tevans.uk@googlemail.com] 
Sent: 01 June 2009 13:50
To: krad
Cc: xorquewasp@googlemail.com; freebsd-hackers@freebsd.org
Subject: RE: Request for opinions - gvinum or ccd?

On Mon, 2009-06-01 at 09:32 +0100, krad wrote:
> Zfs has been designed for highly scalable redundant disk pools therefore
> using it on a single drive kind of goes against it ethos. Remember a lot
of
> the blurb in the man page was written by sun and therefore is written with
> corporates in mind, therefore the cost with of the data vs an extra drive
> being so large why wouldn't you make it redundant.
> 
> Having said that sata drives are cheap these days so you would have to be
on
> the tightest of budgets not to do a mirror.
> 
> Having said all this we quite often us zfs on a single drive, well sort
of.
> The sun clusters have external storage for the shared file systems. These
> are usually a bunch of drives, raid 5, 10 or whatever. Then export a
single
> lun, which is presented to the various nodes. There is a zpool created on
> this LUN. So to all intents and purposes zfs thinks its on a single drive
> (the redundancy provided by the external array). This is common practice
and
> we see no issues with it.

By doing this surely you lose a lot of the self healing that ZFS offers?
For instance, if the underlying vdev is just a raid5, then a disk
failure combined with an undetected checksum error on a different disk
would lead you to lose all your data. Or am I missing something?

(PS, top posting is bad)

Tom


From tevans.uk at googlemail.com  Mon Jun  1 14:15:50 2009
From: tevans.uk at googlemail.com (Tom Evans)
Date: Mon Jun  1 14:15:56 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <20754E8E3F5940E0875AD0BC76A78BAB@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<20090531201445.GA82420@logik.internal.network>
	<alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl>
	<0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl>
	<20090531235943.GA77374@logik.internal.network>
	<3346D963EE2E4D28AE89CB71C2F92939@uk.tiscali.intl>
	<1243860612.9871.68.camel@strangepork.london.mintel.ad>
	<20754E8E3F5940E0875AD0BC76A78BAB@uk.tiscali.intl>
Message-ID: <1243865743.9871.75.camel@strangepork.london.mintel.ad>

On Mon, 2009-06-01 at 14:19 +0100, krad wrote:
> no you would only loose the data for that block. Zfs also checksums meta
> data, but by default keeps multiple copies of it so that's fairly resilient.
> If you had the copies set to > 1 then you wouldn't loose the block either,
> unless you were real unlucky. 
> 
> It's just about pushing the odds back further and further. If you are super
> paranoid by all means put in 48 drive, group them into 5 x 8 drive raidz2
> vdevs, have a bunch of hot spares, and enable copies=5 for blocks and
> metadata, then duplicate the system and put the other box on another
> continent and zfs send all you updates every 15 mins via a private
> deadicated. This will all prove very resilient, but you will get very little
> % storage from your drives, and have quite a large bandwidth bill 8)
> 
> Oh and don't forget the scrub you disk regularly. BTW that would rebuild any
> missing copies as well (eg if you increase the number of copies after data
> is stored on the fs)
> 

Well, no you wouldn't, because ZFS would never get to try to recover
that error. Since that one block is bad, and you lost a disk, your
underlying RAID-5 would not be able to recoverable, and you just lost
the entire contents of the RAID-5. ZFS wouldn't be able to recover
anything from it. The only time ZFS could recover from this scenario is
if you scrubbed before you had your disk failure. Hard to predict disk
failures..

What I'm trying to say (badly) is that this is redundancy that ZFS knows
nothing about, so it cannot recover from it in the same manner that a 5
disk raidz can. If this happened to a 5 disk raid-z, you would lose just
the corrupted block, rather than all your data.

PS, top posting is still bad. Thanks for making me cut the context out
of all these emails.

Cheers

Tom

From wojtek at wojtek.tensor.gdynia.pl  Mon Jun  1 15:50:02 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun  1 15:50:14 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <3ED3189838484C8FBDECE5478A0DDFF4@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><20090531201445.GA82420@logik.internal.network><alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl><0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl><20090531235943.GA77374@logik.internal.network><20090531201408.45dd4334@bhuda.mired.org>
	<alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>
	<442651299CA2445991E65F64AF9E65D7@uk.tiscali.intl>
	<alpine.BSF.2.00.0906011103160.29890@wojtek.tensor.gdynia.pl>
	<3ED3189838484C8FBDECE5478A0DDFF4@uk.tiscali.intl>
Message-ID: <alpine.BSF.2.00.0906011749271.31620@wojtek.tensor.gdynia.pl>

> Its all done on write, so if you update the file it will have multiple
> copies again

what is exactly what i said in the beginning.
From bruce at cran.org.uk  Mon Jun  1 20:28:25 2009
From: bruce at cran.org.uk (Bruce Cran)
Date: Mon Jun  1 20:28:32 2009
Subject: pkg_info segfault Revision: 193189
In-Reply-To: <4A22F7CD.2070500@gmail.com>
References: <4A22F7CD.2070500@gmail.com>
Message-ID: <20090601212819.7fa6598e@gluon.draftnet>

On Sun, 31 May 2009 17:34:05 -0400
Eitan Adler <eitanadlerlist@gmail.com> wrote:

> pkg_info --IwantAcookie
> Segmentation fault: 11 (core dumped)
> on FreeBSD 7.2-STABLE i386

getopt_long expects the array of options to be NULL-terminated, so it's
walking off the end.

-- 
Bruce Cran
-------------- next part --------------
A non-text attachment was scrubbed...
Name: main.c.diff
Type: text/x-patch
Size: 320 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090601/da5dfe8c/main.c.bin
From glen.j.barber at gmail.com  Mon Jun  1 20:30:29 2009
From: glen.j.barber at gmail.com (Glen Barber)
Date: Mon Jun  1 20:30:36 2009
Subject: pkg_info segfault Revision: 193189
In-Reply-To: <20090601212819.7fa6598e@gluon.draftnet>
References: <4A22F7CD.2070500@gmail.com>
	<20090601212819.7fa6598e@gluon.draftnet>
Message-ID: <4ad871310906011330l5a374b27gb4527006c1c47cfa@mail.gmail.com>

On Mon, Jun 1, 2009 at 4:28 PM, Bruce Cran <bruce@cran.org.uk> wrote:
> On Sun, 31 May 2009 17:34:05 -0400
> Eitan Adler <eitanadlerlist@gmail.com> wrote:
>
>> pkg_info --IwantAcookie
>> Segmentation fault: 11 (core dumped)
>> on FreeBSD 7.2-STABLE i386
>
> getopt_long expects the array of options to be NULL-terminated, so it's
> walking off the end.
>

Hi guys,

There was a PR submitted (about pkg_add) already about this, with a
patch.  I am unsure if the patch was looked at yet.

-- 
Glen Barber
FreeBSD Tutorials: http://www.dev-urandom.com
From to.my.trociny at gmail.com  Mon Jun  1 20:33:43 2009
From: to.my.trociny at gmail.com (Mikolaj Golub)
Date: Mon Jun  1 20:33:50 2009
Subject: 7.1-STABLE crash
In-Reply-To: <E1MB0eY-0005z1-00.asmodean-list-ru@f230.mail.ru> (Asmodean
	Dark's message of "Mon\, 01 Jun 2009 10\:05\:46 +0400")
References: <E1MB0eY-0005z1-00.asmodean-list-ru@f230.mail.ru>
Message-ID: <86oct7r8wt.fsf@kopusha.onet>


On Mon, 01 Jun 2009 10:05:46 +0400 Asmodean Dark wrote:

 AD> Hello,

 AD> We have cluster of FreeBSD VPN servers with running mpd3.18 and sometimes it crash:

Do you really need a such outdated version of mpd while mpd5 is available in
ports?

 AD> # uname -a
 AD> FreeBSD vpn 7.1-STABLE FreeBSD 7.1-STABLE #13: Wed Mar 18 14:53:13 YEKT 2009     root@vpn:/usr/src/sys/i386/compile/kconf  i386

 AD> # dmesg
 AD> Fatal trap 12: page fault while in kernel mode
 AD> cpuid = 1; apic id = 01
 AD> fault virtual address   = 0x8
 AD> fault code              = supervisor read, page not present
 AD> instruction pointer     = 0x20:0x806dcb88
 AD> stack pointer           = 0x28:0xe70775c4
 AD> frame pointer           = 0x28:0xe70775e8
 AD> code segment            = base 0x0, limit 0xfffff, type 0x1b
 AD>                         = DPL 0, pres 1, def32 1, gran 1
 AD> processor eflags        = interrupt enabled, resume, IOPL = 0
 AD> current process         = 22 (em0 taskq)
 AD> trap number             = 12

 AD> (kgdb) bt
 AD> #0  0x8063d6b0 in doadump ()
 AD> #1  0x8063dc44 in boot ()
 AD> #2  0x8063e0ca in panic ()
 AD> #3  0x807dab3d in trap_fatal ()
 AD> #4  0x807daeba in trap_pfault ()
 AD> #5  0x807db7bd in trap ()
 AD> #6  0x807c2a3b in calltrap ()
 AD> #7  0x806dcb88 in rn_match ()
 AD> #8  0x806ddc8a in rn_lookup ()
 AD> #9  0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894
 AD> #10 0x80710c3d in ipfw_check_in (arg=0x0, m0=0xe7017700, ifp=0x91c5a800, dir=1, inp=0x0) at ../../../netinet/ip_fw_pfil.c:125
 AD> #11 0x806dc20f in pfil_run_hooks ()
 AD> #12 0x80713984 in ip_input (m=0x91954c00) at ../../../netinet/ip_input.c:416
 AD> #13 0x806ec0d9 in ng_iface_rcvdata ()
 AD> #14 0x806e9570 in ng_apply_item ()
 AD> #15 0x806e8569 in ng_snd_item ()
 AD> #16 0x806e9570 in ng_apply_item ()
 AD> #17 0x806e8569 in ng_snd_item ()
 AD> #18 0x806e9570 in ng_apply_item ()
 AD> #19 0x806e8569 in ng_snd_item ()
 AD> #20 0x806f16a7 in ng_ppp_proto_recv ()
 AD> #21 0x806f3ed2 in ng_ppp_rcvdata ()
 AD> #22 0x806e9570 in ng_apply_item ()
 AD> #23 0x806e8569 in ng_snd_item ()
 AD> #24 0x806e9570 in ng_apply_item ()
 AD> #25 0x806e8569 in ng_snd_item ()
 AD> #26 0x806ee3c3 in ng_ksocket_incoming2 ()
 AD> #27 0x806e969d in ng_apply_item ()
 AD> #28 0x806ea8aa in ngintr ()
 AD> #29 0x806dab72 in swi_net ()
 AD> #30 0x8061e265 in ithread_loop ()
 AD> #31 0x8061adf5 in fork_exit ()
 AD> #32 0x807c2ab0 in fork_trampoline ()

It looks like the kernel crashed when ipfw was looking for the packet's
src/dst in a table. The table might have been corrupted (being modified?)  at
that time. Do you have some automatic blocker or some other script that
periodically add/remove IPs in a ipfw table?

 AD> What can I do with it? Are additional info needed?

It is a bit strange for me that in the bt output the file source information
is displayed only for several functions. Do you have the kernel and all
modules built with the debugging symbols? If you don't I would recommend to
rebuild the kernel so I would be able to provide bt (from the next crash :-)
with all necessary info available. Do the bt output for other crashes looks
the same?

-- 
Mikolaj Golub
From bruce at cran.org.uk  Mon Jun  1 21:04:59 2009
From: bruce at cran.org.uk (Bruce Cran)
Date: Mon Jun  1 21:05:05 2009
Subject: pkg_info segfault Revision: 193189
In-Reply-To: <4ad871310906011330l5a374b27gb4527006c1c47cfa@mail.gmail.com>
References: <4A22F7CD.2070500@gmail.com>
	<20090601212819.7fa6598e@gluon.draftnet>
	<4ad871310906011330l5a374b27gb4527006c1c47cfa@mail.gmail.com>
Message-ID: <20090601220453.7613cd80@gluon.draftnet>

On Mon, 1 Jun 2009 16:30:26 -0400
Glen Barber <glen.j.barber@gmail.com> wrote:

> On Mon, Jun 1, 2009 at 4:28 PM, Bruce Cran <bruce@cran.org.uk> wrote:
> > On Sun, 31 May 2009 17:34:05 -0400
> > Eitan Adler <eitanadlerlist@gmail.com> wrote:
> >
> >> pkg_info --IwantAcookie
> >> Segmentation fault: 11 (core dumped)
> >> on FreeBSD 7.2-STABLE i386
> >
> > getopt_long expects the array of options to be NULL-terminated, so
> > it's walking off the end.
> >
> 
> Hi guys,
> 
> There was a PR submitted (about pkg_add) already about this, with a
> patch.  I am unsure if the patch was looked at yet.
> 

The PR is bin/133473
(http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/133473) and contains the
same patch for pkg_info.

-- 
Bruce Cran
From jchambers at ucla.edu  Tue Jun  2 03:14:27 2009
From: jchambers at ucla.edu (Jason Chambers)
Date: Tue Jun  2 03:14:33 2009
Subject: Monitoring throughput of PCIe lanes
Message-ID: <4A249906.8050802@ucla.edu>

Hello all,

I'm wondering if there is currently a way to monitor the throughput of a
PCIe lane or group of lanes associated with a device ?

I've done a little exploring of the source and man pages but have yet to
find anything that seems to relate in an obvious form.

I wonder if netgraph could somehow be used for this ?


Thanks,

--Jason

From julian at elischer.org  Tue Jun  2 04:31:09 2009
From: julian at elischer.org (Julian Elischer)
Date: Tue Jun  2 04:31:16 2009
Subject: Monitoring throughput of PCIe lanes
In-Reply-To: <4A249906.8050802@ucla.edu>
References: <4A249906.8050802@ucla.edu>
Message-ID: <4A24AB0C.70506@elischer.org>

Jason Chambers wrote:
> Hello all,
> 
> I'm wondering if there is currently a way to monitor the throughput of a
> PCIe lane or group of lanes associated with a device ?
> 
> I've done a little exploring of the source and man pages but have yet to
> find anything that seems to relate in an obvious form.
> 
> I wonder if netgraph could somehow be used for this ?
> 
> 
> Thanks,
> 
> --Jason
> 
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"


unfortunatly PCIe traffic occurs at a level below that at which
the OS can really monitor.

there would have to be some hardware support I think.

THEORETICALLY you could get every device driver to report back
how much traffic it has sent to each device and you could aggregate
that by bus bt it would be a lot of work and overhead
ad wouldn't really be agood measure of burst throughput...
From dnelson at allantgroup.com  Tue Jun  2 05:44:53 2009
From: dnelson at allantgroup.com (Dan Nelson)
Date: Tue Jun  2 05:44:59 2009
Subject: Monitoring throughput of PCIe lanes
In-Reply-To: <4A24AB0C.70506@elischer.org>
References: <4A249906.8050802@ucla.edu> <4A24AB0C.70506@elischer.org>
Message-ID: <20090602052809.GF90077@dan.emsphone.com>

In the last episode (Jun 01), Julian Elischer said:
> Jason Chambers wrote:
> > I'm wondering if there is currently a way to monitor the throughput of a
> > PCIe lane or group of lanes associated with a device ?
> > 
> > I've done a little exploring of the source and man pages but have yet to
> > find anything that seems to relate in an obvious form.
> 
> unfortunatly PCIe traffic occurs at a level below that at which the OS can
> really monitor.
> 
> there would have to be some hardware support I think.

Solaris has a busstat command that can print a huge number of low-level
counters, including PCI DMA counts.  If it's supported under OpenSolaris
it should be easy to check and see whether it's dependant on Sun hardware or
works with any PC (just boot it up and run busstat -l).

-- 
	Dan Nelson
	dnelson@allantgroup.com
From asmodean at list.ru  Tue Jun  2 06:36:34 2009
From: asmodean at list.ru (Asmodean Dark)
Date: Tue Jun  2 06:36:41 2009
Subject: 7.1-STABLE crash
In-Reply-To: <86oct7r8wt.fsf@kopusha.onet>
References: <86oct7r8wt.fsf@kopusha.onet>
Message-ID: <E1MBNbs-0005NL-00.asmodean-list-ru@f47.mail.ru>

>> Do you really need a such outdated version of mpd while mpd5 is available in ports?
Yes, we need mpd3 because there are many our custom patches for it (some RADIUS functions, bugfixes etc).

>> Do you have some automatic blocker or some other script that periodically add/remove IPs in a ipfw table?
Yes. For some our users we use ipfw-based rules (SMTP port blocking, enabling transparent proxy and other). Mpd ifaceup script contain something like this:
	if [ "$FilterId" = "no_transparent" ]; then
		$IPFW table 3 add $IP
	fi

And, in ipfw.rules:
	add allow tcp from table(3) to any dst-port 80 in recv ng*

Why table can be corrupted? ipfw check added address, isn`t it?

>> Do the bt output for other crashes looks the same?
It`s a first dump obtained. There is some problems in receiving it, because all servers is network-booted (via PXE) and have no usable dump devices :) This dump obtained with USB flash device connected to server. Also, crashes is not often.

Thank you
From to.my.trociny at gmail.com  Tue Jun  2 07:46:28 2009
From: to.my.trociny at gmail.com (Mikolaj Golub)
Date: Tue Jun  2 07:46:34 2009
Subject: 7.1-STABLE crash
In-Reply-To: <E1MBNbs-0005NL-00.asmodean-list-ru@f47.mail.ru> (Asmodean Dark's
	message of "Tue\, 02 Jun 2009 10\:36\:32 +0400")
References: <86oct7r8wt.fsf@kopusha.onet>
	<E1MBNbs-0005NL-00.asmodean-list-ru@f47.mail.ru>
Message-ID: <813aajkrhs.fsf@zhuzha.ua1>

On Tue, 02 Jun 2009 10:36:32 +0400 Asmodean Dark wrote:

>>> Do you really need a such outdated version of mpd while mpd5 is available in ports?
> Yes, we need mpd3 because there are many our custom patches for it (some RADIUS functions, bugfixes etc).
>
>>> Do you have some automatic blocker or some other script that periodically add/remove IPs in a ipfw table?
> Yes. For some our users we use ipfw-based rules (SMTP port blocking, enabling transparent proxy and other). Mpd ifaceup script contain something like this:
> 	if [ "$FilterId" = "no_transparent" ]; then
> 		$IPFW table 3 add $IP
> 	fi
>
> And, in ipfw.rules:
> 	add allow tcp from table(3) to any dst-port 80 in recv ng*
>
> Why table can be corrupted? ipfw check added address, isn`t it?

It might be that when ipfw was looking for IP in the table it was being
modified by ifaceup script at that time. ipfw has lock protection so this
thing should not have happened but...

Could you run something like this?

ps -auxl -M /path/to/vmcore -N /path/to/kernel.symbols

We could look if at the moment of the crash some process was running that was
adding/removing ipfw table.

Other good and simple thing is to run crashinfo(8) utility and provide its
output. If it runs flawlessly it should contain ps output too among other
useful information.

>>> Do the bt output for other crashes looks the same?
> It`s a first dump obtained. There is some problems in receiving it, because all servers is network-booted (via PXE) and have no usable dump devices :) This dump obtained with USB flash device connected to server. Also, crashes is not often.

So we could try to get as much info from this dump as we can :-). Could you
post here the whole output of kgdb from its start, not only bt (I mean things
like "Reading symbols from..." and the command line itself)?

Also you can try in kgdb session:

fr 9
list
p *cmd
p cmd->arg1
p a

-- 
Mikolaj Golub
From asmodean at list.ru  Tue Jun  2 09:41:45 2009
From: asmodean at list.ru (Asmodean Dark)
Date: Tue Jun  2 09:41:52 2009
Subject: 7.1-STABLE crash
In-Reply-To: <813aajkrhs.fsf@zhuzha.ua1>
References: <813aajkrhs.fsf@zhuzha.ua1>
Message-ID: <E1MBQV2-0000al-00.asmodean-list-ru@f241.mail.ru>

I`ve attached the requested information. I hope this helps.

-----Original Message-----
From: Mikolaj Golub <to.my.trociny@gmail.com>
To: Asmodean Dark <asmodean@list.ru>
Date: Tue, 02 Jun 2009 10:46:23 +0300
Subject: Re: 7.1-STABLE crash
On Tue, 02 Jun 2009 10:36:32 +0400 Asmodean Dark wrote:

>>> Do you really need a such outdated version of mpd while mpd5 is available in ports?
> Yes, we need mpd3 because there are many our custom patches for it (some RADIUS functions, bugfixes etc).
>
>>> Do you have some automatic blocker or some other script that periodically add/remove IPs in a ipfw table?
> Yes. For some our users we use ipfw-based rules (SMTP port blocking, enabling transparent proxy and other). Mpd ifaceup script contain something like this:
> if [ "$FilterId" = "no_transparent" ]; then
> $IPFW table 3 add $IP
> fi
>
> And, in ipfw.rules:
> add allow tcp from table(3) to any dst-port 80 in recv ng*
>
> Why table can be corrupted? ipfw check added address, isn`t it?

It might be that when ipfw was looking for IP in the table it was being
modified by ifaceup script at that time. ipfw has lock protection so this
thing should not have happened but...

Could you run something like this?

ps -auxl -M /path/to/vmcore -N /path/to/kernel.symbols

We could look if at the moment of the crash some process was running that was
adding/removing ipfw table.

Other good and simple thing is to run crashinfo(8) utility and provide its
output. If it runs flawlessly it should contain ps output too among other
useful information.

>>> Do the bt output for other crashes looks the same?
> It`s a first dump obtained. There is some problems in receiving it, because all servers is network-booted (via PXE) and have no usable dump devices  This dump obtained with USB flash device connected to server. Also, crashes is not often.

So we could try to get as much info from this dump as we can . Could you
post here the whole output of kgdb from its start, not only bt (I mean things
like "Reading symbols from..." and the command line itself)?

Also you can try in kgdb session:

fr 9
list
p *cmd
p cmd->arg1
p a

-------------- next part --------------
A non-text attachment was scrubbed...
Name: crashinfo.tar.bz2
Type: application/octet-stream
Size: 99980 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090602/31e309b5/crashinfo.tar-0001.obj
-------------- next part --------------
# LC_ALL=C ps -auxl -M /tmp/crash/vmcore.0 -N /tmp/crash/kernel.symbols
USER     PID %CPU %MEM   VSZ   RSS  TT  STAT STARTED      TIME COMMAND            UID  PPID CPU PRI NI MWCHAN
root       0  0.0  0.0     0     0  ??  DLs   1Jan70 -28302384:-40.55 [swapper]            0     0   0 -16  0 sched
root       1  0.0  0.0  1888   532  ??  DLs   1Jan70 -22953083:-22.55 [init]               0     0   0   8  0 wait
root       2  0.0  0.0     0     0  ??  DL    1Jan70 -35018924:-59.55 [g_event]            0     0   0  -8  0 -
root       3  0.0  0.0     0     0  ??  DL    1Jan70 19733865:36.00 [g_up]               0     0   0  -8  0 -
root       4  0.0  0.0     0     0  ??  DL    1Jan70 -21084496:00.55 [g_down]             0     0   0  -8  0 -
root       5  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [xpt_thrd]           0     0   0  -8  0 ccb_sc
root       6  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [acpi_task_0]        0     0   0   8  0 -
root       7  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [acpi_task_1]        0     0   0   8  0 -
root       8  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [acpi_task_2]        0     0   0   8  0 -
root       9  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [kqueue taskq]       0     0   0   8  0 -
root      10  0.0  0.0     0     0  ??  RL    1Jan70 22749128:30.00 [idle: cpu1]         0     0   0 171  0 -
root      11  0.0  0.0     0     0  ??  RL    1Jan70 9219592:17.00 [idle: cpu0]         0     0   0 171  0 -
root      12  0.0  0.0     0     0  ??  RL    1Jan70 14087762:45.00 [swi4: clock]        0     0   0 -32  0 -
root      13  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [swi3: vm]           0     0   0 -36  0 -
root      14  0.0  0.0     0     0  ??  RL    1Jan70 17900236:59.00 [swi1: net]          0     0   0 -44  0 -
root      15  0.0  0.0     0     0  ??  DL    1Jan70 -29370189:00.55 [yarrow]             0     0   0  44  0 -
root      16  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [swi6: task queu     0     0   0 -24  0 -
root      17  0.0  0.0     0     0  ??  WL    1Jan70 274009:24.00 [swi2: cambio]       0     0   0 -40  0 -
root      18  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [swi5: +]            0     0   0 -28  0 -
root      19  0.0  0.0     0     0  ??  DL    1Jan70 188:53.00 [thread taskq]       0     0   0   8  0 -
root      20  0.0  0.0     0     0  ??  WL    1Jan70 4600168:46.00 [swi6: Giant tas     0     0   0 -24  0 -
root      21  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [irq9: acpi0]        0     0   0 -52  0 -
root      22  0.0  0.0     0     0  ??  RL    1Jan70 -10575708:-22.55 [em0 taskq]          0     0   0 -68  0 -
root      23  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [irq18: em1 uhci     0     0   0 -68  0 -
root      24  0.0  0.0     0     0  ??  DL    1Jan70 3264118:40.00 [usb0]               0     0   0   8  0 usbevt
root      25  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [usbtask-hc]         0     0   0   8  0 usbtsk
root      26  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [usbtask-dr]         0     0   0   8  0 usbtsk
root      27  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [irq21: uhci1+]      0     0   0 -64  0 -
root      28  0.0  0.0     0     0  ??  DL    1Jan70 3607326:48.00 [usb1]               0     0   0   8  0 usbevt
root      29  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [irq17: uhci2 eh     0     0   0 -64  0 -
root      30  0.0  0.0     0     0  ??  DL    1Jan70 3381869:17.00 [usb2]               0     0   0   8  0 usbevt
root      31  0.0  0.0     0     0  ??  DL    1Jan70 3518333:14.00 [usb3]               0     0   0   8  0 usbevt
root      32  0.0  0.0     0     0  ??  RL    1Jan70 672675:23.00 [irq23: uhci3 eh     0     0   0 -64  0 -
root      33  0.0  0.0     0     0  ??  DL    1Jan70 3256232:07.00 [usb4]               0     0   0   8  0 usbevt
root      34  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [irq19: uhci4]       0     0   0 -64  0 -
root      35  0.0  0.0     0     0  ??  DL    1Jan70 3301012:03.00 [usb5]               0     0   0   8  0 usbevt
root      36  0.0  0.0     0     0  ??  DL    1Jan70 3376947:56.00 [usb6]               0     0   0   8  0 usbevt
root      37  0.0  0.0     0     0  ??  DL    1Jan70 4143595:52.00 [usb7]               0     0   0   8  0 usbevt
root      38  0.0  0.0     0     0  ??  DL    1Jan70   0:00.00 [em1 taskq]          0     0   0 -68  0 -
root      39  0.0  0.0     0     0  ??  WL    1Jan70 17026944:14.00 [irq14: ata0]        0     0   0 -64  0 -
root      40  0.0  0.0     0     0  ??  WL    1Jan70   0:00.00 [irq15: ata1]        0     0   0 -64  0 -
root      41  0.0  0.0     0     0  ??  WL    1Jan70 453060:39.00 [irq1: atkbd0]       0     0   0 -60  0 -
root      42  0.0  0.0     0     0  ??  DL    1Jan70 2888630:00.00 [dummynet]           0     0   0 -68  0 -
root      43  0.0  0.0     0     0  ??  DL    1Jan70 537454:24.00 [md0]                0     0   0  -8  0 mdwait
root      44  0.0  0.0     0     0  ??  DL    1Jan70 -21141059:-47.55 [pagedaemon]         0     0   0 -16  0 psleep
root      45  0.0  0.0     0     0  ??  DL    1Jan70  75:57.00 [vmdaemon]           0     0   0  20  0 psleep
root      46  0.0  0.0     0     0  ??  DL    1Jan70 737245:15.00 [pagezero]           0     0   0 171  0 pgzero
root      47  0.0  0.0     0     0  ??  DL    1Jan70 -5215608:-59.55 [idlepoll]           0     0   0 171  0 pollid
root      48  0.0  0.0     0     0  ??  DL    1Jan70 -35220794:-43.55 [bufdaemon]          0     0   0 -16  0 psleep
root      49  0.0  0.0     0     0  ??  DL    1Jan70 3263896:16.00 [vnlru]              0     0   0  -4  0 vlruwt
root      50  0.0  0.0     0     0  ??  DL    1Jan70 -14008286:-51.55 [syncer]             0     0   0  20  0 syncer
root      51  0.0  0.0     0     0  ??  DL    1Jan70 -7574436:-19.55 [softdepflush]       0     0   0 -16  0 sdflus
root     118  0.0  0.0     0     0  ??  DL    1Jan70 -26345438:-14.55 [md1]                0     0   0  -8  0 mdwait
root     280  0.0  0.0  3088    64  ??  D     1Jan70 5188057:38.00 [dhclient]           0     1   0  48  0 select
_dhcp    302  0.0  0.0  3088    64  ??  Ds    1Jan70 27402661:43.00 [dhclient]          65     1   0  44  0 select
root     396  0.0  0.0     0     0  ??  DL    1Jan70 -30322983:-22.55 [md5]                0     0   0  -8  0 mdwait
root     404  0.0  0.0     0     0  ??  DL    1Jan70 28645466:01.00 [md10]               0     0   0  -8  0 mdwait
root     412  0.0  0.0     0     0  ??  DL    1Jan70 -25819269:-55.55 [md15]               0     0   0  -8  0 mdwait
root     420  0.0  0.0     0     0  ??  DL    1Jan70 273691:15.00 [md20]               0     0   0  -8  0 mdwait
root     510  0.0  0.0     0     0  ??  DL    1Jan70 992311:50.00 [md21]               0     0   0  -8  0 mdwait
root     554  0.0  0.0  3300    32  ??  Ds    1Jan70 -17645368:-45.55 [rpcbind]            0     1   0  44  0 select
root     556  0.0  0.0 266288    16  ??  Ds    1Jan70 -26070283:-44.55 [rpc.statd]          0     1   0  44  0 select
root     569  0.0  0.0  6248    44  ??  Ds    1Jan70 -25412810:-36.55 [rpc.lockd]          0     1   0  44  0 select
daemon   573  0.0  0.0  6248    44  ??  D     1Jan70 9510:40.00 [rpc.lockd]          1   569   0   4  0 nfsloc
root     946  0.0  0.0  3144    32  ??  Rs    1Jan70 27673326:45.00 [syslogd]            0     1   0  44  0 -
quagga   958  0.0  0.0  4812   152  ??  Ds    1Jan70 -7630094:-54.55 [zebra]            101     1   0  44  0 select
quagga   964  0.0  0.0 15176     8  ??  Rs    1Jan70 8082176:22.00 [ospfd]            101     1   0  44  0 -
root    1062  0.0  0.0  7000     4  ??  D     1Jan70 -28023279:-25.55 [perl]               0     1   0   4  0 accept
root    1084  0.0  0.0 65992   232  ??  Rs    1Jan70 -19135518:-47.55 [mpd]                0     1   0  62  0 -
root    1094  0.0  0.0 44728    24  ??  D     1Jan70 31368572:49.00 [snmpd]              0     1   0  44  0 select
nagios  1100  0.0  0.0  3088    20  ??  Ds    1Jan70 30057867:43.00 [nrpe]            1023     1   0   4  0 accept
root    1140  0.0  0.0  5776   392  ??  Ds    1Jan70 -9652890:-8.55 [ntpd]               0     1   0  44  0 select
root    1294  0.0  0.0  4948   188  ??  Ds    1Jan70 34498147:01.55 [sshd]               0     1   0  44  0 select
root    1308  0.0  0.0  3172    32  ??  Ds    1Jan70 31886589:41.00 [cron]               0     1   0   8  0 nanslp
root    1333  0.0  0.0  3420   108  ??  D     1Jan70 33952404:01.55 [sh]                 0     1   0   8  0 wait
root    1401  0.0  0.0  3144    20  ??  Ds+   1Jan70 45271:13.00 [getty]              0     1   0   5  0 ttyin
root    1402  0.0  0.0  3144    20  ??  Ds+   1Jan70 40783:24.00 [getty]              0     1   0   5  0 ttyin
root    1403  0.0  0.0  3144    20  ??  Ds+   1Jan70 43523:33.00 [getty]              0     1   0   5  0 ttyin
root    1404  0.0  0.0  3144    20  ??  Ds+   1Jan70 39453:31.00 [getty]              0     1   0   5  0 ttyin
root    1405  0.0  0.0  3144    20  ??  Ds+   1Jan70 40387:05.00 [getty]              0     1   0   5  0 ttyin
root    1406  0.0  0.0  3144    20  ??  Ds+   1Jan70 39278:45.00 [getty]              0     1   0   5  0 ttyin
root    1407  0.0  0.0  3144    20  ??  Ds+   1Jan70 39826:16.00 [getty]              0     1   0   5  0 ttyin
root    1408  0.0  0.0  3144    20  ??  Ds+   1Jan70 40036:44.00 [getty]              0     1   0   5  0 ttyin
root   69523  0.0  0.0  1336     4  ??  D     1Jan70   0:00.00 [sleep]              0  1333   0   8  0 nanslp
-------------- next part --------------
# kgdb kernel.debug vmcore.0
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...No struct type named linker_file.
No struct type named linker_file.
No struct type named linker_file.
Attempt to extract a component of a value that is not a structure.
No struct type named linker_file.
No struct type named linker_file.
No struct type named linker_file.
Attempt to extract a component of a value that is not a structure.

Attempt to extract a component of a value that is not a structure pointer.
Attempt to extract a component of a value that is not a structure pointer.
Attempt to extract a component of a value that is not a structure pointer.
Attempt to extract a component of a value that is not a structure pointer.
#0  0x8063d6b0 in doadump ()
(kgdb) bt
#0  0x8063d6b0 in doadump ()
#1  0x8063dc44 in boot ()
#2  0x8063e0ca in panic ()
#3  0x807dab3d in trap_fatal ()
#4  0x807daeba in trap_pfault ()
#5  0x807db7bd in trap ()
#6  0x807c2a3b in calltrap ()
#7  0x806dcb88 in rn_match ()
#8  0x806ddc8a in rn_lookup ()
#9  0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894
#10 0x80710c3d in ipfw_check_in (arg=0x0, m0=0xe7017700, ifp=0x91c5a800, dir=1, inp=0x0) at ../../../netinet/ip_fw_pfil.c:125
#11 0x806dc20f in pfil_run_hooks ()
#12 0x80713984 in ip_input (m=0x91954c00) at ../../../netinet/ip_input.c:416
#13 0x806ec0d9 in ng_iface_rcvdata ()
#14 0x806e9570 in ng_apply_item ()
#15 0x806e8569 in ng_snd_item ()
#16 0x806e9570 in ng_apply_item ()
#17 0x806e8569 in ng_snd_item ()
#18 0x806e9570 in ng_apply_item ()
#19 0x806e8569 in ng_snd_item ()
#20 0x806f16a7 in ng_ppp_proto_recv ()
#21 0x806f3ed2 in ng_ppp_rcvdata ()
#22 0x806e9570 in ng_apply_item ()
#23 0x806e8569 in ng_snd_item ()
#24 0x806e9570 in ng_apply_item ()
#25 0x806e8569 in ng_snd_item ()
#26 0x806ee3c3 in ng_ksocket_incoming2 ()
#27 0x806e969d in ng_apply_item ()
#28 0x806ea8aa in ngintr ()
#29 0x806dab72 in swi_net ()
#30 0x8061e265 in ithread_loop ()
#31 0x8061adf5 in fork_exit ()
#32 0x807c2ab0 in fork_trampoline ()
(kgdb) fr 9
#9  0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894
1894            sa.sin_len = 8;
(kgdb) list
1889            struct sockaddr_in sa;
1890
1891            if (tbl >= IPFW_TABLES_MAX)
1892                    return (0);
1893            rnh = ch->tables[tbl];
1894            sa.sin_len = 8;
1895            sa.sin_addr.s_addr = addr;
1896            ent = (struct table_entry *)(rnh->rnh_lookup(&sa, NULL, rnh));
1897            if (ent != NULL) {
1898                    *val = ent->value;
(kgdb) p *cmd
$1 = {opcode = O_IP_SRC_LOOKUP, len = 1 '\001', arg1 = 2}
(kgdb) p cmd->arg1
$2 = 2
(kgdb) p a
Variable "a" is not available.
From rea-fbsd at codelabs.ru  Tue Jun  2 09:52:06 2009
From: rea-fbsd at codelabs.ru (Eygene Ryabinkin)
Date: Tue Jun  2 09:52:13 2009
Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file
	symlink) vulnerability
In-Reply-To: <86r5y7u9r3.fsf@ds4.des.no>
References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no>
	<20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no>
	<20090529210855.V1643@besplex.bde.org>
	<86vdnju9z1.fsf@ds4.des.no> <86r5y7u9r3.fsf@ds4.des.no>
Message-ID: <qlE9VS4n+oBC3U4EHMxYsjC7RHI@XX1fo6zQUfC4h0jjRC6IBz3oNH4>

Dag-Erling, good day.

Fri, May 29, 2009 at 06:58:08PM +0200, Dag-Erling Sm??rgrav wrote:
> Index: sys/kern/vfs_lookup.c
> ===================================================================
> --- sys/kern/vfs_lookup.c	(revision 193028)
> +++ sys/kern/vfs_lookup.c	(working copy)
> @@ -454,7 +454,6 @@
>  	int docache;			/* == 0 do not cache last component */
>  	int wantparent;			/* 1 => wantparent or lockparent flag */
>  	int rdonly;			/* lookup read-only flag bit */
> -	int trailing_slash;
>  	int error = 0;
>  	int dpunlocked = 0;		/* dp has already been unlocked */
>  	struct componentname *cnp = &ndp->ni_cnd;
> @@ -529,12 +528,10 @@
>  	 * trailing slashes to handle symlinks, existing non-directories
>  	 * and non-existing files that won't be directories specially later.
>  	 */
> -	trailing_slash = 0;
>  	while (*cp == '/' && (cp[1] == '/' || cp[1] == '\0')) {
>  		cp++;
>  		ndp->ni_pathlen--;
>  		if (*cp == '\0') {
> -			trailing_slash = 1;
>  			*ndp->ni_next = '\0';	/* XXX for direnter() ... */
>  			cnp->cn_flags |= TRAILINGSLASH;
>  		}
> @@ -711,7 +708,7 @@
>  			error = EROFS;
>  			goto bad;
>  		}
> -		if (*cp == '\0' && trailing_slash &&
> +		if (*cp == '\0' && (cnp->cn_flags & TRAILINGSLASH) &&
>  		     !(cnp->cn_flags & WILLBEDIR)) {

For the current code state, check "*cp == '\0'" seems to be redundant:
VOP_LOOKUP had failed at this point and we're running with EJUSTRETURN.
And EJUSTRETURN can happen only if ISLASTCN is set, that in turn implies
that *ndp->ni_next is 0 and ndp->ni_next is equal to the cp.

Seems like this change makes symlink behavior more consistent: if any
previous component had a trailing slash, then we will bail out unless
we intend to create/rename directory.

By the way, comment before the test "if (rdonly)' seems to be slightly
misleading: we could be not only creating the file but could be renaming
at this point as well.  Perhaps 'creating' should be changed to
'creating/renaming'.

>  			error = ENOENT;
>  			goto bad;
> @@ -788,7 +785,7 @@
>  	 * Check for symbolic link
>  	 */
>  	if ((dp->v_type == VLNK) &&
> -	    ((cnp->cn_flags & FOLLOW) || trailing_slash ||
> +	    ((cnp->cn_flags & FOLLOW) || (cnp->cn_flags & TRAILINGSLASH) ||
>  	     *ndp->ni_next == '/')) {
>  		cnp->cn_flags |= ISSYMLINK;
>  		if (dp->v_iflag & VI_DOOMED) {

Seems like here we can also check for the trailing slash to be set on
any previous invocation (or the current one), since we're following
symlinks in the case of directories, so we should follow them to the
end.

> BTW, what does the "XXX for direnter()" comment mean?

It means that the trailing slashes are eliminated because direnter() for
some FSes may choke on it.  It essentially duplicates the chunk of a big
comment before the block.  'ngp->ni_next' is set to 'cp' before the
block and at that time 'cp' points to a slash that delimits directories
or null character that terminates non-slashed name.

Perhaps 'XXX for direnter()' should be changed to something like
'strip trailing slashes in cnp->cn_nameptr'.

By the way, comment just after 'nextname' label is misleading: we can be
there with symbolic link, but it won't be followed.  So "Not a symbolic
link" can be changed to something like "Not a symbolic link that will
be followed".
-- 
Eygene
 _                ___       _.--.   #
 \`.|\..----...-'`   `-._.-'_.-'`   #  Remember that it is hard
 /  ' `         ,       __.--'      #  to read the on-line manual
 )/' _/     \   `-_,   /            #  while single-stepping the kernel.
 `-'" `"\_  ,_.-;_.-\_ ',  fsc/as   #
     _.-'_./   {_.'   ; /           #    -- FreeBSD Developers handbook
    {_.-``-'         {_/            #
From kraduk at googlemail.com  Tue Jun  2 08:52:21 2009
From: kraduk at googlemail.com (krad)
Date: Tue Jun  2 11:20:09 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <alpine.BSF.2.00.0906011749271.31620@wojtek.tensor.gdynia.pl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><20090531201445.GA82420@logik.internal.network><alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl><0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl><20090531235943.GA77374@logik.internal.network><20090531201408.45dd4334@bhuda.mired.org>
	<alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>
	<442651299CA2445991E65F64AF9E65D7@uk.tiscali.intl>
	<alpine.BSF.2.00.0906011103160.29890@wojtek.tensor.gdynia.pl>
	<3ED3189838484C8FBDECE5478A0DDFF4@uk.tiscali.intl>
	<alpine.BSF.2.00.0906011749271.31620@wojtek.tensor.gdynia.pl>
Message-ID: <D67289F90DB846A68CDA65F9E355FF58@uk.tiscali.intl>

" You need to write a program that will just rewrite all files to make
this."

No you don't you just make sure you scrub the pools regularly once a week
for instance. This way you will hopefully see small block error way before
you have a full drive failure.  

At the end of the day if the data is super critical to you then you should
follow best practices and build an array with sufficient amount of
redundancy in for you needs.

If you cant justify getting 4+ drives to build a raidz2 or raid 10 type
scenario then the data on said array cant actually be worth much itself,
therefore I might be annoying but in the whole scale of things not that
costly if you loose it. If it were you would do thing properly in the first
place



-----Original Message-----
From: Wojciech Puchar [mailto:wojtek@wojtek.tensor.gdynia.pl] 
Sent: 01 June 2009 16:50
To: krad
Cc: 'Mike Meyer'; freebsd-hackers@freebsd.org; xorquewasp@googlemail.com
Subject: RE: Request for opinions - gvinum or ccd?

> Its all done on write, so if you update the file it will have multiple
> copies again

what is exactly what i said in the beginning.

From des at des.no  Tue Jun  2 11:28:37 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Tue Jun  2 11:28:44 2009
Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file
	symlink) vulnerability
In-Reply-To: <qlE9VS4n+oBC3U4EHMxYsjC7RHI@XX1fo6zQUfC4h0jjRC6IBz3oNH4> (Eygene
	Ryabinkin's message of "Tue, 2 Jun 2009 13:51:59 +0400")
References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no>
	<20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no>
	<20090529210855.V1643@besplex.bde.org> <86vdnju9z1.fsf@ds4.des.no>
	<86r5y7u9r3.fsf@ds4.des.no>
	<qlE9VS4n+oBC3U4EHMxYsjC7RHI@XX1fo6zQUfC4h0jjRC6IBz3oNH4>
Message-ID: <86skiiri1p.fsf@ds4.des.no>

Eygene Ryabinkin <rea-fbsd@codelabs.ru> writes:
> For the current code state, check "*cp == '\0'" seems to be redundant:
> [...]  By the way, comment before the test "if (rdonly)' seems to be
> slightly misleading [...]

OK, I see that.  I've removed the check and rewritten the comment.

What about this "temporary assert"?

		/*
		 * This is a temporary assert to make sure I know what the
		 * behavior here was.
		 */
		KASSERT((cnp->cn_flags & (WANTPARENT|LOCKPARENT)) != 0,
		   ("lookup: Unhandled case."));

It was added by jeff@ four years ago, with the following log message:

 - Add a few asserts for some unusual conditions that I do not believe can
   happen.  These will later go away and turn into implementations for these
   conditions.

Any reason not to remove it?

> Seems like here we can also check for the trailing slash to be set on
> any previous invocation (or the current one), since we're following
> symlinks in the case of directories, so we should follow them to the
> end.

I'm not sure I understand...

> Perhaps 'XXX for direnter()' should be changed to something like
> 'strip trailing slashes in cnp->cn_nameptr'.

I'll just remove it, since the previous comment clearly explains what is
going on.

> By the way, comment just after 'nextname' label is misleading: we can be
> there with symbolic link, but it won't be followed.  So "Not a symbolic
> link" can be changed to something like "Not a symbolic link that will
> be followed".

	/*
	 * Not a symbolic link that we will follow.  Continue with the
	 * next component if there is any; otherwise, we're done.
	 */

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From des at des.no  Tue Jun  2 11:31:26 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Tue Jun  2 11:31:32 2009
Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file
	symlink) vulnerability
In-Reply-To: <20090530033902.Q6382@delplex.bde.org> (Bruce Evans's message of
	"Sat, 30 May 2009 04:17:43 +1000 (EST)")
References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no>
	<20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no>
	<20090529210855.V1643@besplex.bde.org> <86vdnju9z1.fsf@ds4.des.no>
	<86r5y7u9r3.fsf@ds4.des.no> <20090530033902.Q6382@delplex.bde.org>
Message-ID: <86oct6rhwz.fsf@ds4.des.no>

Bruce Evans <bde@zeta.org.au> writes:
> This comment could do with some rewording to emphasize inheritance of the
> flag and to improve the grammar of the comment.

Suggestions?  For reference, here's the entire comment:

	/*
	 * Replace multiple slashes by a single slash and trailing slashes
	 * by a null.  This must be done before VOP_LOOKUP() because some
	 * fs's don't know about trailing slashes.  Remember if there were
	 * trailing slashes to handle symlinks, existing non-directories
	 * and non-existing files that won't be directories specially later.
	 */

>> -		if (*cp == '\0' && trailing_slash &&
>> +		if (*cp == '\0' && (cnp->cn_flags & TRAILINGSLASH) &&
>> 		     !(cnp->cn_flags & WILLBEDIR)) {
>> 			error = ENOENT;
>> 			goto bad;
>
> Try replacing *cp == '\0' by (cnp->cn_flags & ISLASTCN) and maybe combine
> the flags tests.  Apparently I hacked in the *cp test because I didn't
> quite understand ISLASTCN.

Is the test necessary at all?  Cf. Eygene's comment.

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From ler at lerctr.org  Tue Jun  2 13:14:10 2009
From: ler at lerctr.org (Larry Rosenman)
Date: Tue Jun  2 13:14:18 2009
Subject: Monitoring throughput of PCIe lanes
In-Reply-To: <20090602052809.GF90077@dan.emsphone.com>
References: <4A249906.8050802@ucla.edu> <4A24AB0C.70506@elischer.org>
	<20090602052809.GF90077@dan.emsphone.com>
Message-ID: <alpine.BSF.2.00.0906020802580.61795@thebighonker.lerctr.org>

On Tue, 2 Jun 2009, Dan Nelson wrote:

> In the last episode (Jun 01), Julian Elischer said:
>> Jason Chambers wrote:
>>> I'm wondering if there is currently a way to monitor the throughput of a
>>> PCIe lane or group of lanes associated with a device ?
>>>
>>> I've done a little exploring of the source and man pages but have yet to
>>> find anything that seems to relate in an obvious form.
>>
>> unfortunatly PCIe traffic occurs at a level below that at which the OS can
>> really monitor.
>>
>> there would have to be some hardware support I think.
>
> Solaris has a busstat command that can print a huge number of low-level
> counters, including PCI DMA counts.  If it's supported under OpenSolaris
> it should be easy to check and see whether it's dependant on Sun hardware or
> works with any PC (just boot it up and run busstat -l).
>
On a SuperMicro board:
root@borg:~# busstat -l
busstat: No devices available in system.
root@borg:~#

That's under OpenSolaris 2008.11

So, I expect it needs hardware support.

>

-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 512-248-2683                 E-Mail: ler@lerctr.org
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  2 15:16:10 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  2 15:16:17 2009
Subject: Synchronize cache failed.
Message-ID: <alpine.BSF.2.00.0906021713170.1314@wojtek.tensor.gdynia.pl>

i bought new digital camera+"8GB" flash card, and when i plug it into USB 
port, it's detected properly

umass0: <OLYMPUS FE20,X15,C25, class 0/0, rev 2.00/1.00, addr 2> on uhub1
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <OLYMPUS FE20,X15,C25 1.00> Removable Direct Access SCSI-0 device
da0: 1.000MB/s transfers
da0: 7646MB (15659008 512 byte sectors: 255H 63S/T 974C)


i can do everything fine, but i'm getting

umass0: Phase Error, residue = 0
(da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0x4, scsi 
status == 0x0


every some time.

so i added a device line to usbdevs, then i edited umass.c

to add

         { USB_VENDOR_OLYMPUS, USB_PRODUCT_OLYMPUS_FE20, RID_WILDCARD,
           UMASS_PROTO_SCSI,
           NO_SYNCHRONIZE_CACHE
         },

to static struct umaumass_devdescrs[]


it compiled, all fine, but when i attached camera again system rebooted.

i was in X session and had tail -f /var/log/messages running, and i was 
able to see a message that my camera is unknown device, for maybe 1 second 
before getting reboot.

No memory dump was performed.


Any idea?
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  2 15:17:42 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  2 15:17:49 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <D67289F90DB846A68CDA65F9E355FF58@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><20090531201445.GA82420@logik.internal.network><alpine.BSF.2.00.0905312355240.26545@wojtek.tensor.gdynia.pl><0229B3BF1BE94C82AA11FD06CBE0BDEF@uk.tiscali.intl><20090531235943.GA77374@logik.internal.network><20090531201408.45dd4334@bhuda.mired.org>
	<alpine.BSF.2.00.0906010224050.27329@wojtek.tensor.gdynia.pl>
	<442651299CA2445991E65F64AF9E65D7@uk.tiscali.intl>
	<alpine.BSF.2.00.0906011103160.29890@wojtek.tensor.gdynia.pl>
	<3ED3189838484C8FBDECE5478A0DDFF4@uk.tiscali.intl>
	<alpine.BSF.2.00.0906011749271.31620@wojtek.tensor.gdynia.pl>
	<D67289F90DB846A68CDA65F9E355FF58@uk.tiscali.intl>
Message-ID: <alpine.BSF.2.00.0906021716150.1314@wojtek.tensor.gdynia.pl>

> this."
>
> No you don't you just make sure you scrub the pools regularly once a week
> for instance.

AGAIN example - i had one drive failed, got it out but recovered all data 
as for everything copies was set to more than one.

Or most data, but those with copies=1 wasn't critical for me.

then i add new drive, then do scrub and missing copies are NOT REBUILD.
From deischen at freebsd.org  Tue Jun  2 15:30:02 2009
From: deischen at freebsd.org (Daniel Eischen)
Date: Tue Jun  2 15:30:09 2009
Subject: Synchronize cache failed.
In-Reply-To: <alpine.BSF.2.00.0906021713170.1314@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906021713170.1314@wojtek.tensor.gdynia.pl>
Message-ID: <Pine.GSO.4.64.0906021121260.8636@sea.ntplx.net>

On Tue, 2 Jun 2009, Wojciech Puchar wrote:

> i bought new digital camera+"8GB" flash card, and when i plug it into USB 
> port, it's detected properly
>
> umass0: <OLYMPUS FE20,X15,C25, class 0/0, rev 2.00/1.00, addr 2> on uhub1
> da0 at umass-sim0 bus 0 target 0 lun 0
> da0: <OLYMPUS FE20,X15,C25 1.00> Removable Direct Access SCSI-0 device
> da0: 1.000MB/s transfers
> da0: 7646MB (15659008 512 byte sectors: 255H 63S/T 974C)
>
>
> i can do everything fine, but i'm getting
>
> umass0: Phase Error, residue = 0
> (da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0x4, scsi status 
> == 0x0
>
>
> every some time.
>
> so i added a device line to usbdevs, then i edited umass.c
>
> to add
>
>        { USB_VENDOR_OLYMPUS, USB_PRODUCT_OLYMPUS_FE20, RID_WILDCARD,
>          UMASS_PROTO_SCSI,
>          NO_SYNCHRONIZE_CACHE
>        },
>
> to static struct umaumass_devdescrs[]
>
>
> it compiled, all fine, but when i attached camera again system rebooted.
>
> i was in X session and had tail -f /var/log/messages running, and i was able 
> to see a message that my camera is unknown device, for maybe 1 second before 
> getting reboot.

Try disabling hal (comment out hald_enable in /etc/rc.conf).
If you need hal for X, try booting without X and hal/dbus.

You might also try UMASS_PROTO_SCSI | UMASS_PROTO_BBB, or
UMASS_PROTO_DEFAULT in your quirk.

-- 
DE
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  2 15:33:25 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  2 15:33:31 2009
Subject: Synchronize cache failed.
In-Reply-To: <Pine.GSO.4.64.0906021121260.8636@sea.ntplx.net>
References: <alpine.BSF.2.00.0906021713170.1314@wojtek.tensor.gdynia.pl>
	<Pine.GSO.4.64.0906021121260.8636@sea.ntplx.net>
Message-ID: <alpine.BSF.2.00.0906021732380.1590@wojtek.tensor.gdynia.pl>

>> able to see a message that my camera is unknown device, for maybe 1 second 
>> before getting reboot.
>
> Try disabling hal (comment out hald_enable in /etc/rc.conf).
> If you need hal for X, try booting without X and hal/dbus.

i don't use hal.

>
> You might also try UMASS_PROTO_SCSI | UMASS_PROTO_BBB, or
> UMASS_PROTO_DEFAULT in your quirk.

OK i will report
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  2 15:42:25 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  2 15:42:32 2009
Subject: Synchronize cache failed.
In-Reply-To: <Pine.GSO.4.64.0906021121260.8636@sea.ntplx.net>
References: <alpine.BSF.2.00.0906021713170.1314@wojtek.tensor.gdynia.pl>
	<Pine.GSO.4.64.0906021121260.8636@sea.ntplx.net>
Message-ID: <alpine.BSF.2.00.0906021741040.1806@wojtek.tensor.gdynia.pl>

>> before getting reboot.
>
> Try disabling hal (comment out hald_enable in /etc/rc.conf).
> If you need hal for X, try booting without X and hal/dbus.
>
> You might also try UMASS_PROTO_SCSI | UMASS_PROTO_BBB, or

PROTO_BBB fixed this - now works properly without errors.

THANK YOU!

how to sent device quirks to developers to be added? by sent-pr ?
From simon at comsys.ntu-kpi.kiev.ua  Tue Jun  2 15:43:00 2009
From: simon at comsys.ntu-kpi.kiev.ua (Andrey Simonenko)
Date: Tue Jun  2 15:43:08 2009
Subject: NFS exports atomic and on-the-fly atomic updates
Message-ID: <20090602152103.GA81350@pm513-1.comsys.ntu-kpi.kiev.ua>

Hello,

Here I want to describe changes that allow to make atomic updates of
NFS exports lists, dynamic on-the-fly atomic updates of NFS exports
lists and improve security of NFS exports.

Solved tasks:
-------------

1. NFS export specifications (spec -- for short) updates are atomic.  NFS
   server's users will not get EACCES for exported file systems or wrong
   access rights while exports list is reloaded.

2. The mountd utility has an option for testing configuration, now one can
   check and see real configuration not loading it into nfsserver.

3. The mountd utility does not load incomplete settings to nfsserver,
   wrong configuration will not allow denied exports.

4. Atomic on-the-fly modifications of NFS export specifications were
   implemented, it is possible to change exports settings dynamically.

5. If some file system is mounted or unmounted, then sighup signal sending
   to mountd is not required, this change removes several race conditions.

6. NFS exports related code is a part of nfsserver/ code, NFS export related
   data can be removed from directories not related to NFS.

Which actions are atomic?
-------------------------

1. Loading export settings from exports(5) file into nfsserver is atomic for
   each exported file system and for all exported file systems.

2. Loading updates into nfsserver is atomic for each exported file system and
   for all exported file systems.

3. If a file system was mounted, then loading export settings for it into
   nfssever is atomic for this file system.

Which actions are not atomic?
-----------------------------

1. Loading export specifications for WebNFS file system and specifying
   WebNFS related settings require more than one system call.

2. Since VFS events such as mounting and unmounting are asynchronous,
   events for all exported and not exported file systems are checked
   as separate system calls.

Since nmount(2) is not used in this implementation and subdirectories exports
are not allowed, it is unlikely that these changes will be accepted.

If absence of insecure subdirectories exports is not a problem, then it
it is possible to support both existent mountd and new API on 7-STABLE
(see nfsserver/nfs_srvsubs.c:nfsrv_fhtovp() function patch).

The mountd utility was completely rewritten, actually the better name
for new utility with new properties would be "nfse".  The single source
file mountd.c was split into three .c and three .h files:  mountd.c
(previous code was rewritten and new code was added), mountd.h (new code),
mountd_conf.c (new code), mountd_conf.h (new code), mountd_xdr.c (previous
code was updated to support new data structure and options), mountd_xdr.h
(previous code).

The analog of kern/vfs_export.c was written from zero and now it is called
nfssever/nfs_export.c.

This version of mountd can be used on modified 7.2 system.
It was tested (except WebNFS related settings) on amd64 and i386 arch.

Support of 8.0 system is possible, it is necessary to modify the
nfs_export.c:nfse_check() function (~60 lines) and add new NFSv4 related
options, but since there are sys/nfssever/ and sys/fs/nfsserver/ that have
functions that call VFS_CHECKEXP and there is activity in NFS development,
it is unclear which arguments nfse_check() should accept.  It is better
to discuss arguments list and semantic of nfse_check() first.  I think this
is the only one function that does not allow to use these changes on 8.0
system.

Available patches have only necessary changes to sys/nfsserver/, sys/kern/,
sys/sys/ and sys/conf/.  Complete changes require removing all NFS exports
related data and code from directories not related to NFS.  Also nfs_pub
structure should be a part of the new nfsserver/nfs_export.c file.
I did not make all these changes (all file systems, all NFS related flags
in mount.h, etc.), just to show the main parts of changes.

The following text contains more or less detail description of changes
(definitely I forgot to mention something here).

Major improvements:
-------------------

* Now all export spec updates are atomic.  mountd uses nfssvc(2) for this
  (the new nfssvc(NFSSVC_EXPORT) call is used).  Now it is safe to reload
  exports files.

* New nfs_export.c file was added to nfsserver/, all API details are
  located in nfs_export.h.  All NFS related flags and structures are part
  of nfsserver.

* Now mountd uses kernel event EVFILT_FS to see mount and umount VFS events.
  The mount(8) utility should not send sighup signal to mountd any more.
  New EVENTHANDLERs were declared: vfs_mount_event and vfs_unmount_event.
  Registered function for these handlers are invoked when a file system is
  mounted or unmounted respectively.  The nfsserver uses these event handlers
  to synchronizes own data with available file systems.  Memory leak was
  removed when an exported file system is unmounted.  Now the nfssever
  understands covered file systems (file system mounted on mount point of
  another file system).  

* The mountd utility has a new option -c, that allows to modify export spec
  on-the-fly.  One can clear, add, update, delete export spec.  All updates
  are atomic.  One commands set works like a transaction with changes,
  it is applied completely or is not applied at all.

* Now mountd has the -t switch: parse configuration files or commands and
  output all settings to stdout.  This option allows to check and see real
  configuration.

Incompatible security changes:
------------------------------

* Now subdirectory export is disallowed.  Subdirectory export does not
  improve security, instead it is the right way for misconfiguration
  (export settings for a subdirectory can be completely unrelated to this
  subdirectory and does not protect access to another parts of exported
  file system).

  The nfsserver exports file systems, not directories, looks like that
  subdirectory export for NFS is too complex or impossible to implement
  completely.  Anyway there is nullfs.

  Having read RFCs, documentation for another NFS implementations and
  thoughts in user groups in Internet I think that this (radical)
  modification will improve security.  This version of mountd has the same
  logic of export rules as nfsserver has.

* Now mountd allows to mount file systems, subdirectories and regular files
  (if the -r flag is on) in exported file systems by default.  The -alldirs
  option became obsolete.

Compatible security changes:
----------------------------

* Ignoring exports files is not safe, since remote users can get wrong access
  rights.  Alternative compatible solution: all exports file must be present,
  a user can specify directory/ and all regular files from the given
  directories will be loaded (any directory can be absent).

* Now if mountd cannot correctly parse export specification for some file
  system, then it does not load anything to nfsserver for this file system.
  Ignoring something in exports file is not safe.

* Now security flavors are per address specification settings in nfsserver
  and mountd.

* In rare cases mountd completely ignore settings in exports file, and does
  not load anything into nfsserver (this can happen if mistake in
  configuration does not allow to finish file parsing).

Updates for mountd:
-------------------

* Now everywhere IPv4 and IPv6 addresses are used, since the kernel knows
  nothing about domain names, netgroups, etc.  Now mountdtab file contains
  only address, MOUNT protocol's procedure EXPORT and DUMP output addresses.
  This removes problems with reverse name resolving, but sometimes entries
  are not removed on unmount (depends on used address).

* Better output for MOUNT protocol's procedure EXPORT: host is an address,
  network is an address with prefix.

* Now mountdtab is parsed more carefully.

* Zone scope index checking was removed for IPv6 addresses, nfsserver does
  not check zone scope index.

* Now mountdtab is saved only when mountd exists, no other program in the base
  system uses this file.  The representation of mountdtab file's content in
  memory was optimized.

* Do not leave PID file if some error occurred and mountd exited.

* Allowed to use loopback addresses in the -h option. (I do not like design
  idea of -h, -p and similar options.)

* Corrected incorrect binding when -p option is not used (nobody saw this
  because this can happen very seldom, but I could reproduce this error).

* Wrong implementation of mask creation when prefix length is given as
  /prefixlength was corrected.

Updates for nfsserver:
----------------------

* Previous nfsserver could access released memory returned by VFS_CHECKEXP.
  New code does have this problem.

Updates for exports(5):
-----------------------

* Added new option -host to allow to use host names and the same netgroups
  names at once.

* Added new option -rw: read-write access.

* Added flag `!' for hosts and networks, this flag means "deny access".

* Added new line "options: ...", right now it is used for global -sec,
  -no_mntproc_dump and -no_mntproc_export options, later it can be used for
  NFSv4.

* Added new option -nospec, that means "this line does not have any address
  specifications".

* Added new option -no_mntproc_dump to disable MOUNT protocol's procedure
  DUMP.

* Added new option -no_mntproc_export to disable MOUNT protocol's procedure
  EXPORT.

* exports(5) says that -o is the only one compatible option.  Actually there
  are others: -root and -r for -maproot, -m for -mask and -n for -network.
  Now mountd logs a warning message if an obsolete option is used.

* Do not allow to use any option between -network and -mask.

* Now #-comment can be anywhere in a line. 

* \xxx octal number can be used in directories names and option's arguments
  for representing an arbitrary character.

* Now it is possible to mix hosts and netgroups with networks:
  "host1 -network=somenetwork host2".

* Now it is possible to change options for particular host/network in one
  line: "-ro -mapall=user1 host1 -mapall=user2 host2" (host2 will inherit
  previous option -ro, but will get new -mapall option).  Since previously
  exports(5) says that options must be given before hosts and networks, this
  change is backward compatible and allows to represent all settings for one
  file system in one logical line.

* Content of exports(5) was simplified and updated.

Open questions and tasks:
-------------------------

* There must be a global solution to check whether it is possible to unload
  a KLD module when no process currently is working with its syscalls.

* Looks like that first argument for nfssvc(2) is not a set of flags any more
  (according to STABLE, CURRENT and NFSv4 implementation).  May be there is
  a sense to make it a value, not flags.

* WebNFS related data in nfsserver is protected when export settings are set
  (in vfs_export.c and new nfs_export.c), but when other parts of nfsserver
  access WebNFS related data no synchronization is performed.

* If a file system cannot be exported in NFS, then there must be some flag
  to indicate this (MNT_NFSEXPORTABLE or something more general, see
  fs/msdosfs/msdosfs_vfsops.c for example).

* Should signals be checked more often in mountd?  Right now signals are
  checked when mountd is waiting for RPC request, or if nfssvc's commands
  transaction timeout occurred.  Previous code has race conditions with
  signals and does too many things disallowed by SUSv3 in signal handlers.

* Should be there any limitation in nfsserver on number of export
  specifications and number of command transactions?

* May be mountd should be renamed to something another, eg. nfse.  NFSv4
  does not use MNT procedure, but still needs utility for configuring
  access rights as I understand.  Also nfse command name is more obvious
  for -c commands, eg. "nfse -c 'add /fs -ro'".  As well /etc/exports can
  be renamed to /etc/nfs.exports, /var/db/mountdtab -> /var/db/nfse.mounts.
  Also such renaming will allow to use mountd and new nfse in 7-STABLE at
  the same time and mount(8) from 7-STABLE will not send SIGHUP to nfse,
  since its PID will be saved in the nfse.pid file.

* netgroup.5 can be moved to src/lib/libc/gen/ or src/share/man/man5, this
  documentation is not part of mountd.

Examples:
---------

1. Correct file:
 
exports file:

options: -no_mntproc_dump
/fs -host 1.1.1.1 -ro 2.2.2.2
/fs -network 10.20.30.40
/home -mapall nobody -network 10/8 -mapall operator -network 20.1/8

"mountd -t" output:

configure: reading file exports
Global options:
    -no_mntproc_dump

Directory /fs
    Export specifications:
	-ro -sec sys -maproot=-2:-2 -host 2.2.2.2
	-rw -sec sys -maproot=-2:-2 -host 1.1.1.1
	-rw -sec sys -maproot=-2:-2 -network 10.0.0.0/8

Directory /home (mount point)
    Export specifications:
	-rw -sec sys -mapall 2:5 -network 20.0.0.0/8
	-rw -sec sys -mapall 65534:65534 -network 10.0.0.0/8

2. Wrong file:

exports:

/fs -ro 1.1.1.1
/fs -network 10/8 -host 1.1.1.1
/home -quiet -ro

"mountd -t" output:

configure: reading file exports
parsing error: exports:2: duplicated address specification 1.1.1.1 was found
in this line
parsing error: exports:2: -host option's argument parsing failed

Directory /fs
    Wrong configuration

Directory /home (mount point)
    File system options:
	-quiet
    Export specifications:
	-ro -sec sys -maproot=-2:-2 (default)

3. Commands testing:

mountd -t -c 'add /fs -ro -mapall nobody -host 1.1.1.1 -network !10/8' \
    -c 'flush /home' -c 'update /usr -ro -mapall operator'

configure: parsing -c commands

Directory /fs
    Commands:
	-c    add -ro -sec sys -mapall 65534:65534 -host 1.1.1.1
	-c    add -ro -sec sys -mapall 65534:65534 -network !10.0.0.0/8

Directory /home (mount point)
    Commands:
	-c  flush

Directory /usr (mount point)
    Commands:
	-c update -ro -sec sys -mapall 2:5 (default)

4. Specifying exports(5) files

mountd /etc/exports /etc/local.exports /usr/local/nfs-export/

Files /etc/exports and /etc/local.exports must be present.
The nfs-export directory can be absent, if it present or will be
present, then all regular files from it are read.

Sources
-------

http://comsys.ntu-kpi.kiev.ua/~simon/nfse/
MD5 (nfse-20090602.tar.bz2) = 8670b95fcfb7a80433afa4db43143418
From deischen at freebsd.org  Tue Jun  2 15:44:49 2009
From: deischen at freebsd.org (Daniel Eischen)
Date: Tue Jun  2 15:45:09 2009
Subject: Synchronize cache failed.
In-Reply-To: <alpine.BSF.2.00.0906021741040.1806@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906021713170.1314@wojtek.tensor.gdynia.pl>
	<Pine.GSO.4.64.0906021121260.8636@sea.ntplx.net>
	<alpine.BSF.2.00.0906021741040.1806@wojtek.tensor.gdynia.pl>
Message-ID: <Pine.GSO.4.64.0906021144120.8699@sea.ntplx.net>

On Tue, 2 Jun 2009, Wojciech Puchar wrote:

>>> before getting reboot.
>> 
>> Try disabling hal (comment out hald_enable in /etc/rc.conf).
>> If you need hal for X, try booting without X and hal/dbus.
>> 
>> You might also try UMASS_PROTO_SCSI | UMASS_PROTO_BBB, or
>
> PROTO_BBB fixed this - now works properly without errors.

Can you also try UMASS_PROTO_DEFAULT?

> THANK YOU!
>
> how to sent device quirks to developers to be added? by sent-pr ?

Yes, filing a PR would be good :-)

-- 
DE
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  2 16:00:18 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  2 16:00:26 2009
Subject: Synchronize cache failed.
In-Reply-To: <Pine.GSO.4.64.0906021144120.8699@sea.ntplx.net>
References: <alpine.BSF.2.00.0906021713170.1314@wojtek.tensor.gdynia.pl>
	<Pine.GSO.4.64.0906021121260.8636@sea.ntplx.net>
	<alpine.BSF.2.00.0906021741040.1806@wojtek.tensor.gdynia.pl>
	<Pine.GSO.4.64.0906021144120.8699@sea.ntplx.net>
Message-ID: <alpine.BSF.2.00.0906021749190.1887@wojtek.tensor.gdynia.pl>

>>> 
>>> You might also try UMASS_PROTO_SCSI | UMASS_PROTO_BBB, or
>> 
>> PROTO_BBB fixed this - now works properly without errors.
>
> Can you also try UMASS_PROTO_DEFAULT?

/usr/src/sys/modules/umass/../../dev/usb/umass.c:607: error: 
'UMASS_PROTO_DEFAULT' undeclared here (not in a function)

i have FreeBSD 7.2-B1
>
>> THANK YOU!
>> 
>> how to sent device quirks to developers to be added? by sent-pr ?
>
> Yes, filing a PR would be good :-)
>
done.
From johndecot at yahoo.com  Tue Jun  2 16:57:46 2009
From: johndecot at yahoo.com (john decot)
Date: Tue Jun  2 16:58:02 2009
Subject: REgarding TOS support in Kernel
Message-ID: <4058.23842.qm@web55408.mail.re4.yahoo.com>

Hi,
?
???? I am intermediate user of freebsd. While building squid with zph support, I got error regarding TOS as follows:

forward.cc: In member function 'void FwdState::dispatch()':
forward.cc:978: error: 'SOL_IP' was not declared in this scope

forward.cc:978: error: 'IP_RECVTOS' was not declared in this scope
forward.cc:982: error: 'IP_PKTOPTIONS' was not declared in this scope
*** Error code 1

?? I have wrote to writer of zph and he suggest me to look for freebsd equivalent socket options calls to retrieve/set the TOS value. I am confused whether this will require kernel compile ? if yes what should we have to enable in kernel conf file.

????? I will appreciate any suggestion and thanks in advance.


John




      
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  2 17:39:08 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  2 17:39:15 2009
Subject: REgarding TOS support in Kernel
In-Reply-To: <4058.23842.qm@web55408.mail.re4.yahoo.com>
References: <4058.23842.qm@web55408.mail.re4.yahoo.com>
Message-ID: <alpine.BSF.2.00.0906021934180.2581@wojtek.tensor.gdynia.pl>

> forward.cc:978: error: 'SOL_IP' was not declared in this scope
>
> forward.cc:978: error: 'IP_RECVTOS' was not declared in this scope
> forward.cc:982: error: 'IP_PKTOPTIONS' was not declared in this scope
> *** Error code 1
>
> ?? I have wrote to writer of zph and he suggest me to look for freebsd equivalent socket options calls to retrieve/set the TOS value. I am confused whether this will require kernel compile ? if yes what should we have to enable in kernel conf file.
i was sure it is but now i'm a but confused

do man telnetd

there is:

      -S tos  Sets the IP type-of-service (TOS) option for the telnet 
connec-
              tion to the value tos, which can be a numeric TOS value or, 
on
              systems that support it, a symbolic TOS name found in the
              /etc/iptos file.


anyway /etc/iptos is absent.

looking at /usr/src/contrib/telnet/telnetd/telnetd.c

search for IP_TOS you will find

#if     defined(IPPROTO_IP) && defined(IP_TOS)
         if (from.ss_family == AF_INET) {
# if    defined(HAS_GETTOS)
                 struct tosent *tp;
                 if (tos < 0 && (tp = gettosbyname("telnet", "tcp")))
                         tos = tp->t_tos;
# endif
                 if (tos < 0)
                         tos = 020;      /* Low Delay bit */
                 if (tos
                    && (setsockopt(0, IPPROTO_IP, IP_TOS,
                                   (char *)&tos, sizeof(tos)) < 0)
                    && (errno != ENOPROTOOPT) )
                         syslog(LOG_WARNING, "setsockopt (IP_TOS): %m");
         }
#endif  /* defined(IPPROTO_IP) && defined(IP_TOS) */

IP_TOS is defined in netinet/in.h

so looks like there is something.

in the same time - man setsockopt says nothing

apropos tos says nothing too.
From ticso at cicely7.cicely.de  Tue Jun  2 18:59:06 2009
From: ticso at cicely7.cicely.de (Bernd Walter)
Date: Tue Jun  2 18:59:14 2009
Subject: close-on-exec alternatives (was: Why kernel kills processes that
	run out of memory instead of just failing memory allocation
	system calls?)
In-Reply-To: <20090529193137.GH67847@elvis.mu.org>
References: <4A14F58F.8000801@rawbw.com>
	<Pine.GSO.4.64.0905202344420.1483@zeno.ucsd.edu>
	<4A1594DA.2010707@rawbw.com> <86ljoig08o.fsf@ds4.des.no>
	<20090528213017.GX67847@elvis.mu.org> <863aaow866.fsf@ds4.des.no>
	<20090529193137.GH67847@elvis.mu.org>
Message-ID: <20090602185850.GR1808@cicely7.cicely.de>

On Fri, May 29, 2009 at 12:31:37PM -0700, Alfred Perlstein wrote:
> * Dag-Erling Sm??rgrav <des@des.no> [090529 02:49] wrote:
> > Alfred Perlstein <alfred@freebsd.org> writes:
> > > Dag-Erling Sm??rgrav <des@des.no> writes:
> > > > Usually, what you see is closer to this:
> > > > 
> > > > if ((pid = fork()) == 0) {
> > > >         for (int fd = 3; fd < getdtablesize(); ++fd)
> > > >                 (void)close(fd);
> > > >         execve(path, argv, envp);
> > > >         _exit(1);
> > > > }
> > >
> > > I'm probably missing something, but couldn't you iterate 
> > > in the parent setting the close-on-exec flag then vfork?
> > 
> > This is an example, Alfred.  Like most examples, it is greatly
> > simplified.  I invite you to peruse the source to find real-world
> > instances of non-trivial fork() / execve() usage.
> 
> It wasn't meant to critisize, just ask a question for the specific
> instance because it made me curious.  I know how bad it can be with
> vfork as I observed a few fixes involving mistaken use of vfork at
> another job.

It is still very interesting, because I currently have a similar problem
and wasn't aware of getdtablesize();
A threaded application which needs to call an external script of unknown
runtime.
I don't have all descriptors under my knowledge, because external libs
might open them.
I also believe there could be a race between retrieving the descriptor
and setting close-on-exec.
The only solution which I found so far is using rfork with RFCFDG.
If I undestand RFCFDG correctly the child process has no descriptors at
all.
This is Ok for me, since I don't need to inherit some.

-- 
B.Walter <bernd@bwct.de> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.
From joerg at britannica.bec.de  Tue Jun  2 19:41:25 2009
From: joerg at britannica.bec.de (Joerg Sonnenberger)
Date: Tue Jun  2 19:41:32 2009
Subject: close-on-exec alternatives (was: Why kernel kills processes
	that run out of memory instead of just failing memory allocation
	system calls?)
In-Reply-To: <20090602185850.GR1808@cicely7.cicely.de>
References: <4A14F58F.8000801@rawbw.com>
	<Pine.GSO.4.64.0905202344420.1483@zeno.ucsd.edu>
	<4A1594DA.2010707@rawbw.com> <86ljoig08o.fsf@ds4.des.no>
	<20090528213017.GX67847@elvis.mu.org> <863aaow866.fsf@ds4.des.no>
	<20090529193137.GH67847@elvis.mu.org>
	<20090602185850.GR1808@cicely7.cicely.de>
Message-ID: <20090602194137.GC12154@britannica.bec.de>

On Tue, Jun 02, 2009 at 08:58:50PM +0200, Bernd Walter wrote:
> It is still very interesting, because I currently have a similar problem
> and wasn't aware of getdtablesize();

Note that many (other) systems provide a much simpler and efficient
function for the above, closefrom(3).

Joerg
From rea-fbsd at codelabs.ru  Tue Jun  2 20:01:07 2009
From: rea-fbsd at codelabs.ru (Eygene Ryabinkin)
Date: Tue Jun  2 20:01:15 2009
Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file
	symlink) vulnerability
In-Reply-To: <86skiiri1p.fsf@ds4.des.no>
References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no>
	<20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no>
	<20090529210855.V1643@besplex.bde.org>
	<86vdnju9z1.fsf@ds4.des.no> <86r5y7u9r3.fsf@ds4.des.no>
	<qlE9VS4n+oBC3U4EHMxYsjC7RHI@XX1fo6zQUfC4h0jjRC6IBz3oNH4>
	<86skiiri1p.fsf@ds4.des.no>
Message-ID: <dl+Tldq1IjLtbwOv9fvp7D68xCc@10Ilc7MfiXA2JVIRVQpZfk7cTQ4>

Tue, Jun 02, 2009 at 01:28:34PM +0200, Dag-Erling Sm??rgrav wrote:
> Eygene Ryabinkin <rea-fbsd@codelabs.ru> writes:
> > For the current code state, check "*cp == '\0'" seems to be redundant:
> > [...]  By the way, comment before the test "if (rdonly)' seems to be
> > slightly misleading [...]
> 
> OK, I see that.  I've removed the check and rewritten the comment.

Thanks.

> What about this "temporary assert"?
> 
> 		/*
> 		 * This is a temporary assert to make sure I know what the
> 		 * behavior here was.
> 		 */
> 		KASSERT((cnp->cn_flags & (WANTPARENT|LOCKPARENT)) != 0,
> 		   ("lookup: Unhandled case."));

It is partly handled at the beginning of lookup:
-----
        wantparent = cnp->cn_flags & (LOCKPARENT | WANTPARENT);
        KASSERT(cnp->cn_nameiop == LOOKUP || wantparent,
            ("CREATE, DELETE, RENAME require LOCKPARENT or WANTPARENT."));
-----
So, only LOOKUP could have both {LOCK,WANT}PARENT to be unset.  And it
seems to be fine, because LOOKUP shouldn't care about parent vnode,
either locked or just referenced.  So this assertion seems to be really
redundant.

> > Seems like here we can also check for the trailing slash to be set on
> > any previous invocation (or the current one), since we're following
> > symlinks in the case of directories, so we should follow them to the
> > end.
> 
> I'm not sure I understand...

I meant the following: you're trading 'trailing_slash' to the
TRAILINGSLASH.  The former is local to this call of lookup(), the latter
persists throughout the namei() invocation, so it could be set by the
previous lookup() calls.  But this seems to be fine: since we were
started to look for directory at some point, we should continue to do
it.

> > Perhaps 'XXX for direnter()' should be changed to something like
> > 'strip trailing slashes in cnp->cn_nameptr'.
> 
> I'll just remove it, since the previous comment clearly explains what is
> going on.

May be it's better to leave the comment, but replace it with more
undestandable one: this instruction is a bit tricky and it makes one to
think what the hell is going on.

> > By the way, comment just after 'nextname' label is misleading: we can be
> > there with symbolic link, but it won't be followed.  So "Not a symbolic
> > link" can be changed to something like "Not a symbolic link that will
> > be followed".
> 
> 	/*
> 	 * Not a symbolic link that we will follow.  Continue with the
> 	 * next component if there is any; otherwise, we're done.
> 	 */

Yes, very good.

Thanks.
-- 
Eygene
 _                ___       _.--.   #
 \`.|\..----...-'`   `-._.-'_.-'`   #  Remember that it is hard
 /  ' `         ,       __.--'      #  to read the on-line manual
 )/' _/     \   `-_,   /            #  while single-stepping the kernel.
 `-'" `"\_  ,_.-;_.-\_ ',  fsc/as   #
     _.-'_./   {_.'   ; /           #    -- FreeBSD Developers handbook
    {_.-``-'         {_/            #
From to.my.trociny at gmail.com  Tue Jun  2 20:25:02 2009
From: to.my.trociny at gmail.com (Mikolaj Golub)
Date: Tue Jun  2 20:25:34 2009
Subject: 7.1-STABLE crash
In-Reply-To: <E1MBQV2-0000al-00.asmodean-list-ru@f241.mail.ru> (Asmodean
	Dark's message of "Tue\, 02 Jun 2009 13\:41\:40 +0400")
References: <813aajkrhs.fsf@zhuzha.ua1>
	<E1MBQV2-0000al-00.asmodean-list-ru@f241.mail.ru>
Message-ID: <86ab4qz8mh.fsf@kopusha.onet>


On Tue, 02 Jun 2009 13:41:40 +0400 Asmodean Dark wrote:

 AD> # kgdb kernel.debug vmcore.0
 AD> GNU gdb 6.1.1 [FreeBSD]
 AD> Copyright 2004 Free Software Foundation, Inc.
 AD> GDB is free software, covered by the GNU General Public License, and you are
 AD> welcome to change it and/or distribute copies of it under certain conditions.
 AD> Type "show copying" to see the conditions.
 AD> There is absolutely no warranty for GDB.  Type "show warranty" for details.
 AD> This GDB was configured as "i386-marcel-freebsd"...No struct type named linker_file.
 AD> No struct type named linker_file.
 AD> No struct type named linker_file.
 AD> Attempt to extract a component of a value that is not a structure.
 AD> No struct type named linker_file.
 AD> No struct type named linker_file.
 AD> No struct type named linker_file.
 AD> Attempt to extract a component of a value that is not a structure.

 AD> Attempt to extract a component of a value that is not a structure pointer.
 AD> Attempt to extract a component of a value that is not a structure pointer.
 AD> Attempt to extract a component of a value that is not a structure pointer.
 AD> Attempt to extract a component of a value that is not a structure pointer.
 AD> #0  0x8063d6b0 in doadump ()
 AD> (kgdb) bt
 AD> #0  0x8063d6b0 in doadump ()
 AD> #1  0x8063dc44 in boot ()
 AD> #2  0x8063e0ca in panic ()
 AD> #3  0x807dab3d in trap_fatal ()
 AD> #4  0x807daeba in trap_pfault ()
 AD> #5  0x807db7bd in trap ()
 AD> #6  0x807c2a3b in calltrap ()
 AD> #7  0x806dcb88 in rn_match ()
 AD> #8  0x806ddc8a in rn_lookup ()
 AD> #9  0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894
 AD> #10 0x80710c3d in ipfw_check_in (arg=0x0, m0=0xe7017700, ifp=0x91c5a800, dir=1, inp=0x0) at ../../../netinet/ip_fw_pfil.c:125
 AD> #11 0x806dc20f in pfil_run_hooks ()
 AD> #12 0x80713984 in ip_input (m=0x91954c00) at ../../../netinet/ip_input.c:416
 AD> #13 0x806ec0d9 in ng_iface_rcvdata ()
 AD> #14 0x806e9570 in ng_apply_item ()
 AD> #15 0x806e8569 in ng_snd_item ()
 AD> #16 0x806e9570 in ng_apply_item ()
 AD> #17 0x806e8569 in ng_snd_item ()
 AD> #18 0x806e9570 in ng_apply_item ()
 AD> #19 0x806e8569 in ng_snd_item ()
 AD> #20 0x806f16a7 in ng_ppp_proto_recv ()
 AD> #21 0x806f3ed2 in ng_ppp_rcvdata ()
 AD> #22 0x806e9570 in ng_apply_item ()
 AD> #23 0x806e8569 in ng_snd_item ()
 AD> #24 0x806e9570 in ng_apply_item ()
 AD> #25 0x806e8569 in ng_snd_item ()
 AD> #26 0x806ee3c3 in ng_ksocket_incoming2 ()
 AD> #27 0x806e969d in ng_apply_item ()
 AD> #28 0x806ea8aa in ngintr ()
 AD> #29 0x806dab72 in swi_net ()
 AD> #30 0x8061e265 in ithread_loop ()
 AD> #31 0x8061adf5 in fork_exit ()
 AD> #32 0x807c2ab0 in fork_trampoline ()
 AD> (kgdb) fr 9
 AD> #9  0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894
 AD> 1894            sa.sin_len = 8;
 AD> (kgdb) list
 AD> 1889            struct sockaddr_in sa;
 AD> 1890
 AD> 1891            if (tbl >= IPFW_TABLES_MAX)
 AD> 1892                    return (0);
 AD> 1893            rnh = ch->tables[tbl];
 AD> 1894            sa.sin_len = 8;
^^^^^^^^^ looks strange. On the line 1894 I expected to see rnh_lookup() call,
which is two lines below. Are you sure your source matches the built kernel?

 AD> 1895            sa.sin_addr.s_addr = addr;
 AD> 1896            ent = (struct table_entry *)(rnh->rnh_lookup(&sa, NULL, rnh));
 AD> 1897            if (ent != NULL) {
 AD> 1898                    *val = ent->value;
 AD> (kgdb) p *cmd
 AD> $1 = {opcode = O_IP_SRC_LOOKUP, len = 1 '\001', arg1 = 2}
 AD> (kgdb) p cmd->arg1
 AD> $2 = 2

It crashed looking for src IP in table 2. But from ps otput I don't see the
process that could modify the table in that time. So the table might have been
corrupted earlier.

Unfortunately, reviewing provided info I don't have any good ideas what might
have caused this. May be other people on the list could help...

Recently I saw some backtrace of the crash in rn_match() too but then it was
pf that was looking for IP in the table. It appeared that the guy was running
ssh brute-force blocker and expiretable, which was run periodically, removed
old entries from the table. He just disabled expiretable and this stopped the
crashes.

Actually some output from crashinfo looks suspicious. zero values for fork()
calls, negative values in vmstat -m output... Does the userland where you were
running crashinfo matched the crushed kernel? Also, does you kernel match
userland on crashed box?

And certainly it would be good to provide backtrace with full debugging info
available :-). Do you remember that debugging symbols for modules are needed
too?
                                       
-- 
Mikolaj Golub
From des at des.no  Wed Jun  3 09:03:47 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Wed Jun  3 09:03:54 2009
Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file
	symlink) vulnerability
In-Reply-To: <dl+Tldq1IjLtbwOv9fvp7D68xCc@10Ilc7MfiXA2JVIRVQpZfk7cTQ4> (Eygene
	Ryabinkin's message of "Wed, 3 Jun 2009 00:01:00 +0400")
References: <23727599.post@talk.nabble.com> <86prdvipwe.fsf@ds4.des.no>
	<20090527233110.E4243@delplex.bde.org> <86r5yaijef.fsf@ds4.des.no>
	<20090529210855.V1643@besplex.bde.org> <86vdnju9z1.fsf@ds4.des.no>
	<86r5y7u9r3.fsf@ds4.des.no>
	<qlE9VS4n+oBC3U4EHMxYsjC7RHI@XX1fo6zQUfC4h0jjRC6IBz3oNH4>
	<86skiiri1p.fsf@ds4.des.no>
	<dl+Tldq1IjLtbwOv9fvp7D68xCc@10Ilc7MfiXA2JVIRVQpZfk7cTQ4>
Message-ID: <86my8pelji.fsf@ds4.des.no>

Eygene Ryabinkin <rea-fbsd@codelabs.ru> writes:
> "Dag-Erling Sm?rgrav" <des@des.no> writes:
> > Eygene Ryabinkin <rea-fbsd@codelabs.ru> writes:
> > > Perhaps 'XXX for direnter()' should be changed to something like
> > > 'strip trailing slashes in cnp->cn_nameptr'.
> > I'll just remove it, since the previous comment clearly explains
> > what is going on.
> May be it's better to leave the comment, but replace it with more
> undestandable one: this instruction is a bit tricky and it makes one to
> think what the hell is going on.

Isn't it clearly described in the preceding comment?  Specifically, by
the first two sentences: "Replace multiple slashes by a single slash and
trailing slashes by a null.  This must be done before VOP_LOOKUP()
because some fs's don't know about trailing slashes."

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From rea-fbsd at codelabs.ru  Wed Jun  3 10:27:15 2009
From: rea-fbsd at codelabs.ru (Eygene Ryabinkin)
Date: Wed Jun  3 10:27:22 2009
Subject: FYI Lighttpd 1.4.23 /kernel (trailing '/' on regular file
	symlink) vulnerability
In-Reply-To: <86my8pelji.fsf@ds4.des.no>
References: <86prdvipwe.fsf@ds4.des.no> <20090527233110.E4243@delplex.bde.org>
	<86r5yaijef.fsf@ds4.des.no> <20090529210855.V1643@besplex.bde.org>
	<86vdnju9z1.fsf@ds4.des.no> <86r5y7u9r3.fsf@ds4.des.no>
	<qlE9VS4n+oBC3U4EHMxYsjC7RHI@XX1fo6zQUfC4h0jjRC6IBz3oNH4>
	<86skiiri1p.fsf@ds4.des.no>
	<dl+Tldq1IjLtbwOv9fvp7D68xCc@10Ilc7MfiXA2JVIRVQpZfk7cTQ4>
	<86my8pelji.fsf@ds4.des.no>
Message-ID: <ZuxGdg11i7+YzQKFs9VOfJU60+4@j4OYE6OL8eALCd4BvSxIfwgoxSc>

Wed, Jun 03, 2009 at 11:03:45AM +0200, Dag-Erling Sm??rgrav wrote:
> Isn't it clearly described in the preceding comment?  Specifically, by
> the first two sentences: "Replace multiple slashes by a single slash and
> trailing slashes by a null.  This must be done before VOP_LOOKUP()
> because some fs's don't know about trailing slashes."

Yes, it is clearly described.  But I started to understand this
description only after asking myself "what ndp->ni_next is doing here
and why do we want to place '\0' to this address"?  I could be a bit
stupid, yeah ;))  But this code snippet can be a bit hard to read for
others as well.  May be not -- can't say for sure.
-- 
Eygene
 _                ___       _.--.   #
 \`.|\..----...-'`   `-._.-'_.-'`   #  Remember that it is hard
 /  ' `         ,       __.--'      #  to read the on-line manual
 )/' _/     \   `-_,   /            #  while single-stepping the kernel.
 `-'" `"\_  ,_.-;_.-\_ ',  fsc/as   #
     _.-'_./   {_.'   ; /           #    -- FreeBSD Developers handbook
    {_.-``-'         {_/            #
From to.my.trociny at gmail.com  Wed Jun  3 12:22:56 2009
From: to.my.trociny at gmail.com (Mikolaj Golub)
Date: Wed Jun  3 12:23:03 2009
Subject: 7.1-STABLE crash
In-Reply-To: <86ab4qz8mh.fsf@kopusha.onet> (Mikolaj Golub's message of "Tue\,
	02 Jun 2009 23\:24\:54 +0300")
References: <813aajkrhs.fsf@zhuzha.ua1>
	<E1MBQV2-0000al-00.asmodean-list-ru@f241.mail.ru>
	<86ab4qz8mh.fsf@kopusha.onet>
Message-ID: <81oct5y09v.fsf@zhuzha.ua1>

On Tue, 02 Jun 2009 23:24:54 +0300 Mikolaj Golub wrote:

> Actually some output from crashinfo looks suspicious. zero values for fork()
> calls, negative values in vmstat -m output... Does the userland where you were
> running crashinfo matched the crushed kernel?

It looks like the strange figures in vmstat output because its '-M' option is
broken. I am observing the same issue on my system. I have rebuild the kernel
and the world of today's RELENG_7 to make sure that this is not the issue with
my kernel not in sync with the userland.

zhuzha:~% uname -a
FreeBSD zhuzha.ua1 7.2-STABLE FreeBSD 7.2-STABLE #18: Wed Jun  3 14:28:49 EEST 2009     root@zhuzha.ua1:/usr/obj/usr/src/sys/DEBUG  i386
zhuzha:~% vmstat -m |head                                  
         Type InUse MemUse HighUse Requests  Size(s)
      ata_dma     3     1K       -        3  128
         GEOM    83     9K       -     1176  16,32,64,128,256,512,1024,2048
       isadev    20     2K       -       20  64
    ar_driver     0     0K       -        6  512,2048
   acd_driver     1     2K       -        1  2048
         cdev    27     4K       -       28  128
       kbdmux     6     9K       -        6  16,128,256,2048,4096
        sigio     8     1K       -        8  32
     filedesc   152    46K       -     1637  16,32,64,128,256,512,1024,2048,4096
zhuzha:~% sudo vmstat -m -M /dev/mem -N /boot/kernel/kernel |head
         Type InUse MemUse HighUse Requests  Size(s)
      ata_dma    -3     0K       -        0  
         GEOM 1124760 -1107K       -  1125936  16,32,64,128,256,512,1024,2048,4096,8192
       isadev   -20     0K       -        0  
    ar_driver 19450   -18K       -    19456  32,64
   acd_driver    -1    -1K       -        0  
         cdev   100    -2K       -      128  16
       kbdmux    -6    -7K       -        0  
        sigio    -8     0K       -        0  
     filedesc 384793  -422K       -   386432  16,32,64,128,1024,2048,4096,8192
zhuzha:~% vmstat -s |head                                        
  2380238 cpu context switches
    27952 device interrupts
   573159 software interrupts
   931165 traps
  9997656 system calls
       56 kernel threads created
     1519  fork() calls
       45 vfork() calls
        0 rfork() calls
        0 swap pager pageins
zhuzha:~% sudo vmstat -s -M /dev/mem -N /boot/kernel/kernel |head
        0 cpu context switches
        0 device interrupts
        0 software interrupts
        0 traps
        0 system calls
        0 kernel threads created
        0  fork() calls
        0 vfork() calls
        0 rfork() calls
        0 swap pager pageins

On CURRENT vmstat -m works ok but vmstat -s gives zero values:

fbsd# uname -a
FreeBSD fbsd.zhuzha.ua1 8.0-CURRENT FreeBSD 8.0-CURRENT #8 r193242M: Mon Jun  1 23:43:06 EEST 2009     root@zhuzha.ua1:/home/golub/freebsd/build/obj/home/golub/freebsd/src/sys/GENERIC  i386
fbsd# vmstat -m |head
         Type InUse MemUse HighUse Requests  Size(s)
         cdev    11     2K       -       11  128
CAM dev queue     1     1K       -        1  64
        sigio     1     1K       -        1  32
     filedesc    49    13K       -     1109  16,256,512
         kenv   106     8K       -      109  16,32,64,128,4096
       kqueue     0     0K       -       12  128,1024
    proc-args    29     2K       -      593  16,32,64,128
      ithread    63     6K       -       63  16,64,128
       acpica   426    23K       -     8140  16,32,64,128,256,512,1024
fbsd# vmstat -m -M /dev/mem -N /boot/kernel/kernel |head
         Type InUse MemUse HighUse Requests  Size(s)
         cdev    11     2K       -       11  128
CAM dev queue     1     1K       -        1  64
        sigio     1     1K       -        1  32
     filedesc    49    13K       -     1111  16,256,512
         kenv   106     8K       -      109  16,32,64,128,4096
       kqueue     0     0K       -       12  128,1024
    proc-args    29     2K       -      599  16,32,64,128
      ithread    63     6K       -       63  16,64,128
       acpica   426    23K       -     8140  16,32,64,128,256,512,1024
fbsd# vmstat -s |head
    88380 cpu context switches
     5058 device interrupts
    16851 software interrupts
   138908 traps
   140549 system calls
       18 kernel threads created
     1070  fork() calls
       22 vfork() calls
        0 rfork() calls
        0 swap pager pageins
fbsd# vmstat -s -M /dev/mem -N /boot/kernel/kernel |head
        0 cpu context switches
        0 device interrupts
        0 software interrupts
        0 traps
        0 system calls
        0 kernel threads created
        0  fork() calls
        0 vfork() calls
        0 rfork() calls
        0 swap pager pageins

On 6.3-RELEASE vmstat -s gives zero values only for some counters:

fbsd6# uname -a
FreeBSD fbsd6.zhuzha.ua1 6.3-RELEASE FreeBSD 6.3-RELEASE #0: Wed Jan 16 04:18:52 UTC 2008     root@dessler.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
fbsd6# vmstat -m |head
         Type InUse MemUse HighUse Requests  Size(s)
        DEVFS    12     1K       -       13  16,128
    pfs_nodes    20     3K       -       20  128
         GEOM    52     6K       -      259  16,32,64,128,256,512,1024,2048,4096
       isadev    17     2K       -       17  64
    CAM queue     3     1K       -        3  16
CAM dev queue     1     1K       -        1  64
  ata_generic     2     2K       -        2  1024
         cdev    24     3K       -       24  128
    file desc    64    17K       -      685  16,256,512
fbsd6# vmstat -m -M /dev/mem -N /boot/kernel/kernel |head
         Type InUse MemUse HighUse Requests  Size(s)
        DEVFS    12     1K       -       13  16,128
    pfs_nodes    20     3K       -       20  128
         GEOM    52     6K       -      259  16,32,64,128,256,512,1024,2048,4096
       isadev    17     2K       -       17  64
    CAM queue     3     1K       -        3  16
CAM dev queue     1     1K       -        1  64
  ata_generic     2     2K       -        2  1024
         cdev    24     3K       -       24  128
    file desc    64    17K       -      687  16,256,512
fbsd6# vmstat -s
    70467 cpu context switches
    23614 device interrupts
     2136 software interrupts
    46411 traps
    83608 system calls
       34 kernel threads created
      654  fork() calls
       22 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
      431 vnode pager pageins
     3015 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
       34 pages reactivated
    18811 copy-on-write faults
      116 copy-on-write optimized faults
    13147 zero fill pages zeroed
    13017 zero fill pages prezeroed
        3 intransit blocking page faults
    44963 total VM faults taken
        0 pages affected by kernel thread creation
    52028 pages affected by  fork()
     7493 pages affected by vfork()
        0 pages affected by rfork()
    34465 pages freed
        0 pages freed by daemon
    22467 pages freed by exiting processes
     2172 pages active
     2264 pages inactive
        0 pages in VM cache
     2800 pages wired down
     6528 pages free
     4096 bytes per page
    23942 total name lookups
          cache hits (90% pos + 5% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%
fbsd6# vmstat -s -M /dev/mem -N /boot/kernel/kernel
    76024 cpu context switches
        0 device interrupts
     2144 software interrupts
        0 traps
        0 system calls
       34 kernel threads created
      654  fork() calls
       23 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
      431 vnode pager pageins
     3015 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
       34 pages reactivated
    18826 copy-on-write faults
      116 copy-on-write optimized faults
    13186 zero fill pages zeroed
    13056 zero fill pages prezeroed
        3 intransit blocking page faults
    45035 total VM faults taken
        0 pages affected by kernel thread creation
    52028 pages affected by  fork()
     7967 pages affected by vfork()
        0 pages affected by rfork()
    34518 pages freed
        0 pages freed by daemon
    22502 pages freed by exiting processes
     2177 pages active
     2263 pages inactive
        0 pages in VM cache
     2800 pages wired down
     6524 pages free
     4096 bytes per page
    23983 total name lookups
          cache hits (90% pos + 5% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

-- 
Mikolaj Golub
From tatsiana.severyna at gmail.com  Wed Jun  3 13:17:08 2009
From: tatsiana.severyna at gmail.com (Tatsiana Severyna)
Date: Wed Jun  3 15:56:41 2009
Subject: SoC 2009: puffs port for FreeBSD
Message-ID: <a096b1430906030548w389cebefy6a9553ba07ff11f3@mail.gmail.com>

Hi,

I'm Tatsiana Severyna from Belarus. I'll be working on porting puffs
from NetBSD as a part of this year's summer of code program. refuse
library (fuse compatibility layer on top of puffs) will also be ported
to FreeBSD as during the summer.  My mentor is Konstantin Belousov.

Detailed description is available on wiki:
http://wiki.freebsd.org/SOC2009TatsianaSeveryna
From kostjn at peterhost.ru  Thu Jun  4 09:22:13 2009
From: kostjn at peterhost.ru (Menshikov Konstantin)
Date: Thu Jun  4 09:22:19 2009
Subject: Inline function (difficult  debug)
Message-ID: <4A2792B7.5010006@peterhost.ru>

Hi.
My system FreeBSD 7.1 RELEASE periodically freeze.

I`m compiling kernel with WITNESS and get backtrace.

#0 doadump () at pcpu.h:195
#1 0xffffffff801a899c in db_fncall (dummy1=Variable "dummy1" is not 
available.
) at /usr/src/sys/ddb/db_command.c:516
#2 0xffffffff801a8ecf in db_command (last_cmdp=0xffffffff80825688, 
cmd_table=0x0, dopager=1)
at /usr/src/sys/ddb/db_command.c:413
#3 0xffffffff801a90e0 in db_command_loop () at 
/usr/src/sys/ddb/db_command.c:466
#4 0xffffffff801aaa19 in db_trap (type=Variable "type" is not available.
) at /usr/src/sys/ddb/db_main.c:228
#5 0xffffffff803d5e55 in kdb_trap (type=3, code=0, 
tf=0xffffffffabf284c0) at /usr/src/sys/kern/subr_kdb.c:524
#6 0xffffffff8059a0e5 in trap (frame=0xffffffffabf284c0) at 
/usr/src/sys/amd64/amd64/trap.c:538
#7 0xffffffff8057f60e in calltrap () at 
/usr/src/sys/amd64/amd64/exception.S:209
#8 0xffffffff803d602d in kdb_enter_why (why=0xffffffff80628137 
"witness", msg=0xa <Address 0xa out of bounds>)
at cpufunc.h:63
#9 0xffffffff803e70c3 in witness_checkorder (lock=Variable "lock" is not 
available.
) at /usr/src/sys/kern/subr_witness.c:1126
#10 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff8084fad0, 
opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
) at /usr/src/sys/kern/kern_mutex.c:227
#11 0xffffffff80319838 in sc_puts (scp=0xffffffff8084f980, 
buf=0xffffffffabf286a7 "a@\az\200????", len=1)
at /usr/src/sys/dev/syscons/syscons.c:2519
#12 0xffffffff8031b3da in sc_cnputc (cd=Variable "cd" is not available.
) at /usr/src/sys/dev/syscons/syscons.c:1561
#13 0xffffffff803f9d89 in cnputc (c=97) at /usr/src/sys/kern/tty_cons.c:632
#14 0xffffffff803d993b in putchar (c=97, arg=Variable "arg" is not 
available.
) at /usr/src/sys/kern/subr_prf.c:421
#15 0xffffffff803d7f22 in kvprintf (fmt=0xffffffff80628b89 "cquiring 
duplicate lock of same type: \"%s\"\n",
func=0xffffffff803d98d0 <putchar>, arg=0xffffffffabf28890, radix=10, 
ap=Variable "ap" is not available.
) at /usr/src/sys/kern/subr_prf.c:674
#16 0xffffffff803d9384 in printf (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/subr_prf.c:314
#17 0xffffffff803e705d in witness_checkorder (lock=0xffffffff808668f8, 
flags=0,
file=0xffffffff80626cf8 "/usr/src/sys/kern/subr_sleepqueue.c", line=232) 
at /usr/src/sys/kern/subr_witness.c:948
#18 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff808668f8, 
opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
) at /usr/src/sys/kern/kern_mutex.c:227
#19 0xffffffff803b2271 in wakeup (ident=0xffffffff80850ac0) at 
/usr/src/sys/kern/kern_synch.c:341
#20 0xffffffff803aed95 in tdsignal (p=0xffffff00017978f0, 
td=0xffffff000146a6e0, sig=14, ksi=Variable "ksi" is not available.
)
at /usr/src/sys/kern/kern_sig.c:2292
#21 0xffffffff803ba2de in realitexpire (arg=Variable "arg" is not available.
) at /usr/src/sys/kern/kern_time.c:669
#22 0xffffffff803bbe1a in softclock (dummy=Variable "dummy" is not 
available.
) at /usr/src/sys/kern/kern_timeout.c:274
#23 0xffffffff8038c120 in ithread_loop (arg=0xffffff00010ebc00) at 
/usr/src/sys/kern/kern_intr.c:1088
#24 0xffffffff8038978a in fork_exit (callout=0xffffffff8038c040 
<ithread_loop>, arg=0xffffff00010ebc00,
frame=0xffffffffabf28c80) at /usr/src/sys/kern/kern_fork.c:804
#25 0xffffffff8057f9de in fork_trampoline () at 
/usr/src/sys/amd64/amd64/exception.S:455

But file /usr/src/sys/kern/kern_sig.c:2292
hasn`t run function wake up on string 2292
I`m think, what compiler use inline function, therefore code and trace 
differ.
Tell me please, how compile kernel without any inline function.

-- 
Menshikov Konstantin

From ed at 80386.nl  Thu Jun  4 09:38:33 2009
From: ed at 80386.nl (Ed Schouten)
Date: Thu Jun  4 09:38:45 2009
Subject: Clang: now available from a SVN server near you!
Message-ID: <20090604093831.GE48776@hoeg.nl>

Good news everyone!

As I mentioned at BSDCan, I was going to import my FreeBSD+Clang branch
into SVN. Tuesday I finally had some time to do it, so here's the
result:

	http://svn.freebsd.org/viewvc/base/projects/clangbsd/

You can now build your very own version of FreeBSD with Clang installed
as /usr/bin/cc as follows:

- Check out the clangbsd branch from our SVN repository:
  svn co svn://svn.freebsd.org/base/projects/clangbsd

- Put this in /etc/src.conf:
  WITH_CLANG=
  WITH_CLANG_IS_CC=
  NO_WERROR=
  WERROR=

- Just run `make buildworld' and `make installworld'.

You probably don't want to install it on top of your running system. I
strongly advise you to create a separate jail to do all your testing
with. When using a jail, you probably want to add NO_FSCHG= to
/etc/src.conf as well to keep `make installworld' happy.

So far we've only done testing on amd64 and i386. A lot of ports are
probably still broken. Caveat emptor. Beware of dog. Slippery when wet.

-- 
 Ed Schouten <ed@80386.nl>
 WWW: http://80386.nl/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090604/d457b4b6/attachment.pgp
From rdivacky at FreeBSD.org  Thu Jun  4 11:19:10 2009
From: rdivacky at FreeBSD.org (Roman Divacky)
Date: Thu Jun  4 11:19:17 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <20090604093831.GE48776@hoeg.nl>
References: <20090604093831.GE48776@hoeg.nl>
Message-ID: <20090604111801.GA91410@freebsd.org>

On Thu, Jun 04, 2009 at 11:38:31AM +0200, Ed Schouten wrote:
> Good news everyone!
> 
> As I mentioned at BSDCan, I was going to import my FreeBSD+Clang branch
> into SVN. Tuesday I finally had some time to do it, so here's the
> result:
> 
> 	http://svn.freebsd.org/viewvc/base/projects/clangbsd/
> 
> You can now build your very own version of FreeBSD with Clang installed
> as /usr/bin/cc as follows:
> 
> - Check out the clangbsd branch from our SVN repository:
>   svn co svn://svn.freebsd.org/base/projects/clangbsd
> 
> - Put this in /etc/src.conf:
>   WITH_CLANG=
>   WITH_CLANG_IS_CC=
>   NO_WERROR=
>   WERROR=
> 
> - Just run `make buildworld' and `make installworld'.

it should be able to compile a bootable kernel on amd64/i386 as well.. so
feel free to test that too :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090604/dbfd7301/attachment.pgp
From jhb at freebsd.org  Thu Jun  4 12:22:20 2009
From: jhb at freebsd.org (John Baldwin)
Date: Thu Jun  4 12:22:26 2009
Subject: Inline function (difficult  debug)
In-Reply-To: <4A2792B7.5010006@peterhost.ru>
References: <4A2792B7.5010006@peterhost.ru>
Message-ID: <200906040749.17206.jhb@freebsd.org>

On Thursday 04 June 2009 5:24:07 am Menshikov Konstantin wrote:
> Hi.
> My system FreeBSD 7.1 RELEASE periodically freeze.
> 
> I`m compiling kernel with WITNESS and get backtrace.
> 
> #0 doadump () at pcpu.h:195
> #1 0xffffffff801a899c in db_fncall (dummy1=Variable "dummy1" is not 
> available.
> ) at /usr/src/sys/ddb/db_command.c:516
> #2 0xffffffff801a8ecf in db_command (last_cmdp=0xffffffff80825688, 
> cmd_table=0x0, dopager=1)
> at /usr/src/sys/ddb/db_command.c:413
> #3 0xffffffff801a90e0 in db_command_loop () at 
> /usr/src/sys/ddb/db_command.c:466
> #4 0xffffffff801aaa19 in db_trap (type=Variable "type" is not available.
> ) at /usr/src/sys/ddb/db_main.c:228
> #5 0xffffffff803d5e55 in kdb_trap (type=3, code=0, 
> tf=0xffffffffabf284c0) at /usr/src/sys/kern/subr_kdb.c:524
> #6 0xffffffff8059a0e5 in trap (frame=0xffffffffabf284c0) at 
> /usr/src/sys/amd64/amd64/trap.c:538
> #7 0xffffffff8057f60e in calltrap () at 
> /usr/src/sys/amd64/amd64/exception.S:209
> #8 0xffffffff803d602d in kdb_enter_why (why=0xffffffff80628137 
> "witness", msg=0xa <Address 0xa out of bounds>)
> at cpufunc.h:63
> #9 0xffffffff803e70c3 in witness_checkorder (lock=Variable "lock" is not 
> available.
> ) at /usr/src/sys/kern/subr_witness.c:1126
> #10 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff8084fad0, 
> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
> ) at /usr/src/sys/kern/kern_mutex.c:227
> #11 0xffffffff80319838 in sc_puts (scp=0xffffffff8084f980, 
> buf=0xffffffffabf286a7 "a@\az\200????", len=1)
> at /usr/src/sys/dev/syscons/syscons.c:2519
> #12 0xffffffff8031b3da in sc_cnputc (cd=Variable "cd" is not available.
> ) at /usr/src/sys/dev/syscons/syscons.c:1561
> #13 0xffffffff803f9d89 in cnputc (c=97) at /usr/src/sys/kern/tty_cons.c:632
> #14 0xffffffff803d993b in putchar (c=97, arg=Variable "arg" is not 
> available.
> ) at /usr/src/sys/kern/subr_prf.c:421
> #15 0xffffffff803d7f22 in kvprintf (fmt=0xffffffff80628b89 "cquiring 
> duplicate lock of same type: \"%s\"\n",
> func=0xffffffff803d98d0 <putchar>, arg=0xffffffffabf28890, radix=10, 
> ap=Variable "ap" is not available.
> ) at /usr/src/sys/kern/subr_prf.c:674
> #16 0xffffffff803d9384 in printf (fmt=Variable "fmt" is not available.
> ) at /usr/src/sys/kern/subr_prf.c:314
> #17 0xffffffff803e705d in witness_checkorder (lock=0xffffffff808668f8, 
> flags=0,
> file=0xffffffff80626cf8 "/usr/src/sys/kern/subr_sleepqueue.c", line=232) 
> at /usr/src/sys/kern/subr_witness.c:948
> #18 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff808668f8, 
> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
> ) at /usr/src/sys/kern/kern_mutex.c:227
> #19 0xffffffff803b2271 in wakeup (ident=0xffffffff80850ac0) at 
> /usr/src/sys/kern/kern_synch.c:341
> #20 0xffffffff803aed95 in tdsignal (p=0xffffff00017978f0, 
> td=0xffffff000146a6e0, sig=14, ksi=Variable "ksi" is not available.
> )
> at /usr/src/sys/kern/kern_sig.c:2292
> #21 0xffffffff803ba2de in realitexpire (arg=Variable "arg" is not available.
> ) at /usr/src/sys/kern/kern_time.c:669
> #22 0xffffffff803bbe1a in softclock (dummy=Variable "dummy" is not 
> available.
> ) at /usr/src/sys/kern/kern_timeout.c:274
> #23 0xffffffff8038c120 in ithread_loop (arg=0xffffff00010ebc00) at 
> /usr/src/sys/kern/kern_intr.c:1088
> #24 0xffffffff8038978a in fork_exit (callout=0xffffffff8038c040 
> <ithread_loop>, arg=0xffffff00010ebc00,
> frame=0xffffffffabf28c80) at /usr/src/sys/kern/kern_fork.c:804
> #25 0xffffffff8057f9de in fork_trampoline () at 
> /usr/src/sys/amd64/amd64/exception.S:455
> 
> But file /usr/src/sys/kern/kern_sig.c:2292
> hasn`t run function wake up on string 2292
> I`m think, what compiler use inline function, therefore code and trace 
> differ.
> Tell me please, how compile kernel without any inline function.

makeoptions DEBUG="-g -fno-inline"

However. I think you already have enough information to debug this.  Can you 
go to frame 17 and do an 'l'?  I think the two things you want to print out 
are "i", "*lock", "*lock1", and "*lock2".

-- 
John Baldwin
From rdivacky at FreeBSD.org  Thu Jun  4 12:39:05 2009
From: rdivacky at FreeBSD.org (Roman Divacky)
Date: Thu Jun  4 12:39:20 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
References: <20090604093831.GE48776@hoeg.nl>
	<31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
Message-ID: <20090604123801.GA34971@freebsd.org>

On Thu, Jun 04, 2009 at 02:35:56PM +0200, Erik Cederstrand wrote:
> Den 04/06/2009 kl. 11.38 skrev Ed Schouten:
> 
> >You can now build your very own version of FreeBSD with Clang  
> >installed
> >as /usr/bin/cc as follows:
> 
> Thanks for your hard work, Ed. This is great news!
> 
> You might want to mention that a few parts are still GCC-compiled due  
> to bugs in Clang ( see http://wiki.freebsd.org/ 
> BuildingFreeBSDWithClang). Also, it's very encouraging that the ports  
> run you did with Erwin 
> (http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005274.html ) 
> compiles over 7000 ports with Clang.
> 
> I've asked in the Clang list, but I'd like an opinion from FreeBSD  
> folks, too: Clang supports LTO 
> (http://llvm.org/docs/LinkTimeOptimization.html ), which has a potential 
> for performance improvements. It doesn't work  on FreeBSD because the 
> linker we have (in binutils) doesn't know about  the libLTO that LLVM 
> provides. There's a new linker from GNU called  Gold, but as far as I know 
> it's GPLv3 licensed and therefore  undesirable at least to have in base. 
> LLVM provides a linker (http://llvm.org/cmds/llvm-ld.html ) but "it doesn't 
> interact correctly with conventional nm/ar/etc" 
> (http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005296.html ). 
> There's the ELF toolchain project (elftoolchain.sourceforge.net/)  but a 
> BSD-licensed ld hasn't been developed yet.
> 
> What would be the best way to get LTO to work on FreeBSD?

you could use llvm-ld (see the wiki for instructions how to do it). there's also some 
effort to make gnu ld usable with llvm LTO and I guess the patch could be backported 
to our ld. I guess
From kostjn at peterhost.ru  Thu Jun  4 13:00:53 2009
From: kostjn at peterhost.ru (Menshikov Konstantin)
Date: Thu Jun  4 13:01:01 2009
Subject: Inline function (difficult  debug)
In-Reply-To: <200906040749.17206.jhb@freebsd.org>
References: <4A2792B7.5010006@peterhost.ru>
	<200906040749.17206.jhb@freebsd.org>
Message-ID: <4A27C5E8.7020902@peterhost.ru>

John Baldwin wrote:
> On Thursday 04 June 2009 5:24:07 am Menshikov Konstantin wrote:
>   
>> Hi.
>> My system FreeBSD 7.1 RELEASE periodically freeze.
>>
>> I`m compiling kernel with WITNESS and get backtrace.
>>
>> #0 doadump () at pcpu.h:195
>> #1 0xffffffff801a899c in db_fncall (dummy1=Variable "dummy1" is not 
>> available.
>> ) at /usr/src/sys/ddb/db_command.c:516
>> #2 0xffffffff801a8ecf in db_command (last_cmdp=0xffffffff80825688, 
>> cmd_table=0x0, dopager=1)
>> at /usr/src/sys/ddb/db_command.c:413
>> #3 0xffffffff801a90e0 in db_command_loop () at 
>> /usr/src/sys/ddb/db_command.c:466
>> #4 0xffffffff801aaa19 in db_trap (type=Variable "type" is not available.
>> ) at /usr/src/sys/ddb/db_main.c:228
>> #5 0xffffffff803d5e55 in kdb_trap (type=3, code=0, 
>> tf=0xffffffffabf284c0) at /usr/src/sys/kern/subr_kdb.c:524
>> #6 0xffffffff8059a0e5 in trap (frame=0xffffffffabf284c0) at 
>> /usr/src/sys/amd64/amd64/trap.c:538
>> #7 0xffffffff8057f60e in calltrap () at 
>> /usr/src/sys/amd64/amd64/exception.S:209
>> #8 0xffffffff803d602d in kdb_enter_why (why=0xffffffff80628137 
>> "witness", msg=0xa <Address 0xa out of bounds>)
>> at cpufunc.h:63
>> #9 0xffffffff803e70c3 in witness_checkorder (lock=Variable "lock" is not 
>> available.
>> ) at /usr/src/sys/kern/subr_witness.c:1126
>> #10 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff8084fad0, 
>> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
>> ) at /usr/src/sys/kern/kern_mutex.c:227
>> #11 0xffffffff80319838 in sc_puts (scp=0xffffffff8084f980, 
>> buf=0xffffffffabf286a7 "a@\az\200????", len=1)
>> at /usr/src/sys/dev/syscons/syscons.c:2519
>> #12 0xffffffff8031b3da in sc_cnputc (cd=Variable "cd" is not available.
>> ) at /usr/src/sys/dev/syscons/syscons.c:1561
>> #13 0xffffffff803f9d89 in cnputc (c=97) at /usr/src/sys/kern/tty_cons.c:632
>> #14 0xffffffff803d993b in putchar (c=97, arg=Variable "arg" is not 
>> available.
>> ) at /usr/src/sys/kern/subr_prf.c:421
>> #15 0xffffffff803d7f22 in kvprintf (fmt=0xffffffff80628b89 "cquiring 
>> duplicate lock of same type: \"%s\"\n",
>> func=0xffffffff803d98d0 <putchar>, arg=0xffffffffabf28890, radix=10, 
>> ap=Variable "ap" is not available.
>> ) at /usr/src/sys/kern/subr_prf.c:674
>> #16 0xffffffff803d9384 in printf (fmt=Variable "fmt" is not available.
>> ) at /usr/src/sys/kern/subr_prf.c:314
>> #17 0xffffffff803e705d in witness_checkorder (lock=0xffffffff808668f8, 
>> flags=0,
>> file=0xffffffff80626cf8 "/usr/src/sys/kern/subr_sleepqueue.c", line=232) 
>> at /usr/src/sys/kern/subr_witness.c:948
>> #18 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff808668f8, 
>> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
>> ) at /usr/src/sys/kern/kern_mutex.c:227
>> #19 0xffffffff803b2271 in wakeup (ident=0xffffffff80850ac0) at 
>> /usr/src/sys/kern/kern_synch.c:341
>> #20 0xffffffff803aed95 in tdsignal (p=0xffffff00017978f0, 
>> td=0xffffff000146a6e0, sig=14, ksi=Variable "ksi" is not available.
>> )
>> at /usr/src/sys/kern/kern_sig.c:2292
>> #21 0xffffffff803ba2de in realitexpire (arg=Variable "arg" is not available.
>> ) at /usr/src/sys/kern/kern_time.c:669
>> #22 0xffffffff803bbe1a in softclock (dummy=Variable "dummy" is not 
>> available.
>> ) at /usr/src/sys/kern/kern_timeout.c:274
>> #23 0xffffffff8038c120 in ithread_loop (arg=0xffffff00010ebc00) at 
>> /usr/src/sys/kern/kern_intr.c:1088
>> #24 0xffffffff8038978a in fork_exit (callout=0xffffffff8038c040 
>> <ithread_loop>, arg=0xffffff00010ebc00,
>> frame=0xffffffffabf28c80) at /usr/src/sys/kern/kern_fork.c:804
>> #25 0xffffffff8057f9de in fork_trampoline () at 
>> /usr/src/sys/amd64/amd64/exception.S:455
>>
>> But file /usr/src/sys/kern/kern_sig.c:2292
>> hasn`t run function wake up on string 2292
>> I`m think, what compiler use inline function, therefore code and trace 
>> differ.
>> Tell me please, how compile kernel without any inline function.
>>     
>
> makeoptions DEBUG="-g -fno-inline"
>
> However. I think you already have enough information to debug this.  Can you 
> go to frame 17 and do an 'l'?  I think the two things you want to print out 
> are "i", "*lock", "*lock1", and "*lock2".
>
>   
Unfortunately, the kernel is not compiled with an option-fno-inline.
cc -c -O2 -frename-registers -pipe -fno-strict-aliasing  -std=c99 -g 
-fno-inline -Wall -Wredundant-decls -Wnested-externs 
-Wstrict-prototypes  -Wmissing-prototypes -Wpointer-arith -Winline 
-Wcast-qual  -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc  
-I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL 
-DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common 
-finline-limit=8000 --param inline-unit-growth=100 --param 
large-function-growth=1000  -fno-omit-frame-pointer -mcmodel=kernel 
-mno-red-zone  -mfpmath=387 -mno-sse -mno-sse2 -mno-mmx -mno-3dnow  
-msoft-float -fno-asynchronous-unwind-tables -ffreestanding -Werror  
/usr/src/sys/cam/cam_periph.c
cc1: warnings being treated as errors
/usr/src/sys/cam/cam_periph.c: In function 'scsi_extract_sense':
/usr/src/sys/cam/scsi/scsi_all.h:1215: warning: function 
'scsi_extract_sense' can never be inlined because it is suppressed using 
-fno-inline

I`m try build with -O0 options.

lock in frame 17
(kgdb) print {struct mtx} 0xffffffff808668f8
$4 = {lock_object = {lo_name = 0xffffffff806270eb "sleepq chain", 
lo_type = 0xffffffff806270eb "sleepq chain",
    lo_flags = 720896, lo_witness_data = {lod_list = {stqe_next = 
0xffffffff808793e0}, lod_witness = 0xffffffff808793e0}},
  mtx_lock = 4, mtx_recurse = 0}
next in list
(kgdb) print {struct mtx} 0xffffffff808793e0
$5 = {lock_object = {lo_name = 0xffffffff806270eb "sleepq chain", 
lo_type = 0xffffffff807e8860 "\031\006b\200????\n",
    lo_flags = 2156368928, lo_witness_data = {lod_list = {stqe_next = 
0xffffffff80879420},
      lod_witness = 0xffffffff80879420}}, mtx_lock = 
18446744071571216096, mtx_recurse = 2153913763}
(kgdb) print {struct mtx} 0xffffffff80879420
next
$6 = {lock_object = {lo_name = 0xffffffff806205df "process slock", 
lo_type = 0xffffffff807e8860 "\031\006b\200????\n",
    lo_flags = 2156368992, lo_witness_data = {lod_list = {stqe_next = 
0xffffffff80879460},
      lod_witness = 0xffffffff80879460}}, mtx_lock = 
18446744071571216160, mtx_recurse = 2153889520}

As far as I understand, 2 times are requested blocking of one class.
Besides, too great values mtx_recurse it is wrong.





From unixmania at gmail.com  Thu Jun  4 13:21:26 2009
From: unixmania at gmail.com (Carlos A. M. dos Santos)
Date: Thu Jun  4 13:21:33 2009
Subject: looking for style(9) configuration for Eclipse
Message-ID: <e71790db0906040621s6938f352hdbe3903bf2f0e36e@mail.gmail.com>

Is there any document describing how to configure Eclipse for style(9)
compliance?

Eclipse is the official IDE of the project I'm working on. We use code
taken from FreeBSD, making some changes that my company plans to
contribute back, so I'd like to keep the formatting compliant to
style(9). The Developers' Handbook only mentions Emacs, however.

Thanks in advance

-- 
My preferred quotation of Robert Louis Stevenson is "You cannot
make an omelette without breaking eggs". Not because I like the
omelettes, but because I like the sound of eggs being broken.
From kostjn at peterhost.ru  Thu Jun  4 13:51:57 2009
From: kostjn at peterhost.ru (Menshikov Konstantin)
Date: Thu Jun  4 13:52:24 2009
Subject: Inline function (difficult  debug)
In-Reply-To: <4A27C5E8.7020902@peterhost.ru>
References: <4A2792B7.5010006@peterhost.ru>	<200906040749.17206.jhb@freebsd.org>
	<4A27C5E8.7020902@peterhost.ru>
Message-ID: <4A27D1FC.1020408@peterhost.ru>

Menshikov Konstantin wrote:
> John Baldwin wrote:
>> On Thursday 04 June 2009 5:24:07 am Menshikov Konstantin wrote:
>>  
>>> Hi.
>>> My system FreeBSD 7.1 RELEASE periodically freeze.
>>>
>>> I`m compiling kernel with WITNESS and get backtrace.
>>>
>>> #0 doadump () at pcpu.h:195
>>> #1 0xffffffff801a899c in db_fncall (dummy1=Variable "dummy1" is not 
>>> available.
>>> ) at /usr/src/sys/ddb/db_command.c:516
>>> #2 0xffffffff801a8ecf in db_command (last_cmdp=0xffffffff80825688, 
>>> cmd_table=0x0, dopager=1)
>>> at /usr/src/sys/ddb/db_command.c:413
>>> #3 0xffffffff801a90e0 in db_command_loop () at 
>>> /usr/src/sys/ddb/db_command.c:466
>>> #4 0xffffffff801aaa19 in db_trap (type=Variable "type" is not 
>>> available.
>>> ) at /usr/src/sys/ddb/db_main.c:228
>>> #5 0xffffffff803d5e55 in kdb_trap (type=3, code=0, 
>>> tf=0xffffffffabf284c0) at /usr/src/sys/kern/subr_kdb.c:524
>>> #6 0xffffffff8059a0e5 in trap (frame=0xffffffffabf284c0) at 
>>> /usr/src/sys/amd64/amd64/trap.c:538
>>> #7 0xffffffff8057f60e in calltrap () at 
>>> /usr/src/sys/amd64/amd64/exception.S:209
>>> #8 0xffffffff803d602d in kdb_enter_why (why=0xffffffff80628137 
>>> "witness", msg=0xa <Address 0xa out of bounds>)
>>> at cpufunc.h:63
>>> #9 0xffffffff803e70c3 in witness_checkorder (lock=Variable "lock" is 
>>> not available.
>>> ) at /usr/src/sys/kern/subr_witness.c:1126
>>> #10 0xffffffff8039e341 in _mtx_lock_spin_flags 
>>> (m=0xffffffff8084fad0, opts=0, file=dwarf2_read_address: Corrupted 
>>> DWARF expression.
>>> ) at /usr/src/sys/kern/kern_mutex.c:227
>>> #11 0xffffffff80319838 in sc_puts (scp=0xffffffff8084f980, 
>>> buf=0xffffffffabf286a7 "a@\az\200????", len=1)
>>> at /usr/src/sys/dev/syscons/syscons.c:2519
>>> #12 0xffffffff8031b3da in sc_cnputc (cd=Variable "cd" is not available.
>>> ) at /usr/src/sys/dev/syscons/syscons.c:1561
>>> #13 0xffffffff803f9d89 in cnputc (c=97) at 
>>> /usr/src/sys/kern/tty_cons.c:632
>>> #14 0xffffffff803d993b in putchar (c=97, arg=Variable "arg" is not 
>>> available.
>>> ) at /usr/src/sys/kern/subr_prf.c:421
>>> #15 0xffffffff803d7f22 in kvprintf (fmt=0xffffffff80628b89 "cquiring 
>>> duplicate lock of same type: \"%s\"\n",
>>> func=0xffffffff803d98d0 <putchar>, arg=0xffffffffabf28890, radix=10, 
>>> ap=Variable "ap" is not available.
>>> ) at /usr/src/sys/kern/subr_prf.c:674
>>> #16 0xffffffff803d9384 in printf (fmt=Variable "fmt" is not available.
>>> ) at /usr/src/sys/kern/subr_prf.c:314
>>> #17 0xffffffff803e705d in witness_checkorder 
>>> (lock=0xffffffff808668f8, flags=0,
>>> file=0xffffffff80626cf8 "/usr/src/sys/kern/subr_sleepqueue.c", 
>>> line=232) at /usr/src/sys/kern/subr_witness.c:948
>>> #18 0xffffffff8039e341 in _mtx_lock_spin_flags 
>>> (m=0xffffffff808668f8, opts=0, file=dwarf2_read_address: Corrupted 
>>> DWARF expression.
>>> ) at /usr/src/sys/kern/kern_mutex.c:227
>>> #19 0xffffffff803b2271 in wakeup (ident=0xffffffff80850ac0) at 
>>> /usr/src/sys/kern/kern_synch.c:341
>>> #20 0xffffffff803aed95 in tdsignal (p=0xffffff00017978f0, 
>>> td=0xffffff000146a6e0, sig=14, ksi=Variable "ksi" is not available.
>>> )
>>> at /usr/src/sys/kern/kern_sig.c:2292
>>> #21 0xffffffff803ba2de in realitexpire (arg=Variable "arg" is not 
>>> available.
>>> ) at /usr/src/sys/kern/kern_time.c:669
>>> #22 0xffffffff803bbe1a in softclock (dummy=Variable "dummy" is not 
>>> available.
>>> ) at /usr/src/sys/kern/kern_timeout.c:274
>>> #23 0xffffffff8038c120 in ithread_loop (arg=0xffffff00010ebc00) at 
>>> /usr/src/sys/kern/kern_intr.c:1088
>>> #24 0xffffffff8038978a in fork_exit (callout=0xffffffff8038c040 
>>> <ithread_loop>, arg=0xffffff00010ebc00,
>>> frame=0xffffffffabf28c80) at /usr/src/sys/kern/kern_fork.c:804
>>> #25 0xffffffff8057f9de in fork_trampoline () at 
>>> /usr/src/sys/amd64/amd64/exception.S:455
>>>
>>> But file /usr/src/sys/kern/kern_sig.c:2292
>>> hasn`t run function wake up on string 2292
>>> I`m think, what compiler use inline function, therefore code and 
>>> trace differ.
>>> Tell me please, how compile kernel without any inline function.
>>>     
>>
>> makeoptions DEBUG="-g -fno-inline"
>>
>> However. I think you already have enough information to debug this.  
>> Can you go to frame 17 and do an 'l'?  I think the two things you 
>> want to print out are "i", "*lock", "*lock1", and "*lock2".
>>
>>   
> Unfortunately, the kernel is not compiled with an option-fno-inline.
> cc -c -O2 -frename-registers -pipe -fno-strict-aliasing  -std=c99 -g 
> -fno-inline -Wall -Wredundant-decls -Wnested-externs 
> -Wstrict-prototypes  -Wmissing-prototypes -Wpointer-arith -Winline 
> -Wcast-qual  -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc  
> -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL 
> -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common 
> -finline-limit=8000 --param inline-unit-growth=100 --param 
> large-function-growth=1000  -fno-omit-frame-pointer -mcmodel=kernel 
> -mno-red-zone  -mfpmath=387 -mno-sse -mno-sse2 -mno-mmx -mno-3dnow  
> -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -Werror  
> /usr/src/sys/cam/cam_periph.c
> cc1: warnings being treated as errors
> /usr/src/sys/cam/cam_periph.c: In function 'scsi_extract_sense':
> /usr/src/sys/cam/scsi/scsi_all.h:1215: warning: function 
> 'scsi_extract_sense' can never be inlined because it is suppressed 
> using -fno-inline
>
> I`m try build with -O0 options.
>
> lock in frame 17
> (kgdb) print {struct mtx} 0xffffffff808668f8
> $4 = {lock_object = {lo_name = 0xffffffff806270eb "sleepq chain", 
> lo_type = 0xffffffff806270eb "sleepq chain",
>    lo_flags = 720896, lo_witness_data = {lod_list = {stqe_next = 
> 0xffffffff808793e0}, lod_witness = 0xffffffff808793e0}},
>  mtx_lock = 4, mtx_recurse = 0}
> next in list
> (kgdb) print {struct mtx} 0xffffffff808793e0
> $5 = {lock_object = {lo_name = 0xffffffff806270eb "sleepq chain", 
> lo_type = 0xffffffff807e8860 "\031\006b\200????\n",
>    lo_flags = 2156368928, lo_witness_data = {lod_list = {stqe_next = 
> 0xffffffff80879420},
>      lod_witness = 0xffffffff80879420}}, mtx_lock = 
> 18446744071571216096, mtx_recurse = 2153913763}
> (kgdb) print {struct mtx} 0xffffffff80879420
> next
> $6 = {lock_object = {lo_name = 0xffffffff806205df "process slock", 
> lo_type = 0xffffffff807e8860 "\031\006b\200????\n",
>    lo_flags = 2156368992, lo_witness_data = {lod_list = {stqe_next = 
> 0xffffffff80879460},
>      lod_witness = 0xffffffff80879460}}, mtx_lock = 
> 18446744071571216160, mtx_recurse = 2153889520}
>
> As far as I understand, 2 times are requested blocking of one class.
> Besides, too great values mtx_recurse it is wrong.
>
Sorry, I was mistaken.
thread spin lock.
$35 = {lock_object = {lo_name = 0xffffffff806270eb "sleepq chain", 
lo_type = 0xffffffff806270eb "sleepq chain",
    lo_flags = 720896, lo_witness_data = {lod_list = {stqe_next = 
0xffffffff808793e0}, lod_witness = 0xffffffff808793e0}},
  mtx_lock = 4, mtx_recurse = 0}
(kgdb) print {struct witness} 0xffffffff808793e0
$36 = {w_name = 0xffffffff806270eb "sleepq chain", w_class = 
0xffffffff807e8860, w_list = {stqe_next = 0xffffffff80879420},
  w_typelist = {stqe_next = 0xffffffff80879420}, w_children = 
0xffffffff808b96e0,
  w_file = 0xffffffff80621da3 "/usr/src/sys/kern/kern_sig.c", w_line = 
2291, w_level = 0, w_refcount = 129,
  w_Giant_squawked = 0 '\0', w_other_squawked = 1 '\001', 
w_same_squawked = 1 '\001', w_displayed = 0 '\0'}
next struct witness
proc spin lock
(kgdb) print {struct proc}   0xffffff00013a4478   
$55 = {p_list = {le_next = 0xffffff00013a7000, le_prev = 
0xffffff0001416000}, p_threads = {tqh_first = 0xffffff000144e370,
    tqh_last = 0xffffff000144e380}, p_upcalls = {tqh_first = 0x0, 
tqh_last = 0xffffff00013a4498}, p_slock = {lock_object = {
      lo_name = 0xffffffff806205df "process slock", lo_type = 
0xffffffff806205df "process slock", lo_flags = 720896,
      lo_witness_data = {lod_list = {stqe_next = 0xffffffff80879420}, 
lod_witness = 0xffffffff80879420}},
    mtx_lock = 18446742974215689072, mtx_recurse = 0}...
$56 = {w_name = 0xffffffff806205df "process slock", w_class = 
0xffffffff807e8860, w_list = {
    stqe_next = 0xffffffff80879460}, w_typelist = {stqe_next = 
0xffffffff80879460}, w_children = 0xffffffff808b9720,
  w_file = 0xffffffff8061bef0 "/usr/src/sys/kern/kern_clock.c", w_line = 
281, w_level = 0, w_refcount = 431,
  w_Giant_squawked = 0 '\0', w_other_squawked = 0 '\0', w_same_squawked 
= 0 '\0', w_displayed = 0 '\0'}

If it is fair, while I do not know, where exactly a problem.
If you are ready to try to help, I can give a kernel and dump to memory.

From erdgeist at erdgeist.org  Thu Jun  4 14:27:32 2009
From: erdgeist at erdgeist.org (Dirk Engling)
Date: Thu Jun  4 14:27:39 2009
Subject: Jails, loopback interfaces and sendmail
Message-ID: <4A27D38B.6040108@erdgeist.org>

Dear fellow hackers,

since jail can be bound on multiple IP addresses I tend to clone
multiple loopback interfaces and add one loopback address to each jail

cloned_interfaces="lo1 lo2 lo3"
ifconfig_lo1_alias0="inet 127.0.0.2 netmask 0xffffffff"
ifconfig_lo2_alias0="inet 127.0.0.3 netmask 0xffffffff"
ifconfig_lo3_alias0="inet 127.0.0.4 netmask 0xffffffff"
..

no this is not yet optimal, since I can not run several jails on a
single external IP anymore, but at least local daemons are not visible
to the outside world, anymore.

However, grep -R 127.0.0.1 /etc reveals, that sendmail in many places
assumes localhost to be on 127.0.0.1 instead of looking it up in
/etc/hosts or using 127.0.0.0/8 to identify a local connection.

I worry that more programmers made those assumptions, possibly breaking
more tools.

My question is: Who's the right guy to beg to fix sendmail or
alternatively would it be smart to allow each jail to have its own
concept of 127.0.0.1 on a dummy interface mapped to all jails, that
itself doesn't count as a bound IP address (thus allowing the jail to
bind to an already bound ip address) and is not routed between jails?

Any ideas?

  erdgeist
From wojtek at wojtek.tensor.gdynia.pl  Thu Jun  4 15:18:08 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Thu Jun  4 15:18:15 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <4A27D38B.6040108@erdgeist.org>
References: <4A27D38B.6040108@erdgeist.org>
Message-ID: <alpine.BSF.2.00.0906041717130.34191@wojtek.tensor.gdynia.pl>

> However, grep -R 127.0.0.1 /etc reveals, that sendmail in many places
> assumes localhost to be on 127.0.0.1 instead of looking it up in
> /etc/hosts or using 127.0.0.0/8 to identify a local connection.

calling 127.0.0.1 from jail always loops back within jail. it's all fine.

>
> I worry that more programmers made those assumptions, possibly breaking
> more tools.
>
> My question is: Who's the right guy to beg to fix sendmail or
> alternatively would it be smart to allow each jail to have its own
> concept of 127.0.0.1 on a dummy interface mapped to all jails, that

it already have
From marius at nuenneri.ch  Thu Jun  4 15:47:00 2009
From: marius at nuenneri.ch (=?ISO-8859-1?Q?Marius_N=FCnnerich?=)
Date: Thu Jun  4 15:47:14 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <20090604093831.GE48776@hoeg.nl>
References: <20090604093831.GE48776@hoeg.nl>
Message-ID: <b649e5e0906040819k5bf8c16y317783b7b560fc1c@mail.gmail.com>

Thanks to the team for this!

On Thu, Jun 4, 2009 at 11:38, Ed Schouten <ed@80386.nl> wrote:
> Good news everyone!
>
> As I mentioned at BSDCan, I was going to import my FreeBSD+Clang branch
> into SVN. Tuesday I finally had some time to do it, so here's the
> result:
>
> ? ? ? ?http://svn.freebsd.org/viewvc/base/projects/clangbsd/
>
> You can now build your very own version of FreeBSD with Clang installed
> as /usr/bin/cc as follows:
>
> - Check out the clangbsd branch from our SVN repository:
> ?svn co svn://svn.freebsd.org/base/projects/clangbsd

If one has a (recent) head checkout one can also use the faster and lighter
svn switch svn://svn.freebsd.org/base/projects/clangbsd
in the head working directory (less traffic for the server too).
From erik at cederstrand.dk  Thu Jun  4 12:36:00 2009
From: erik at cederstrand.dk (Erik Cederstrand)
Date: Thu Jun  4 16:08:15 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <20090604093831.GE48776@hoeg.nl>
References: <20090604093831.GE48776@hoeg.nl>
Message-ID: <31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>

Den 04/06/2009 kl. 11.38 skrev Ed Schouten:

> You can now build your very own version of FreeBSD with Clang  
> installed
> as /usr/bin/cc as follows:

Thanks for your hard work, Ed. This is great news!

You might want to mention that a few parts are still GCC-compiled due  
to bugs in Clang ( see http://wiki.freebsd.org/ 
BuildingFreeBSDWithClang). Also, it's very encouraging that the ports  
run you did with Erwin (http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005274.html 
) compiles over 7000 ports with Clang.

I've asked in the Clang list, but I'd like an opinion from FreeBSD  
folks, too: Clang supports LTO (http://llvm.org/docs/LinkTimeOptimization.html 
), which has a potential for performance improvements. It doesn't work  
on FreeBSD because the linker we have (in binutils) doesn't know about  
the libLTO that LLVM provides. There's a new linker from GNU called  
Gold, but as far as I know it's GPLv3 licensed and therefore  
undesirable at least to have in base. LLVM provides a linker (http://llvm.org/cmds/llvm-ld.html 
) but "it doesn't interact correctly with conventional nm/ar/etc" (http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005296.html 
). There's the ELF toolchain project (elftoolchain.sourceforge.net/)  
but a BSD-licensed ld hasn't been developed yet.

What would be the best way to get LTO to work on FreeBSD?

Thanks,
Erik
From glen.j.barber at gmail.com  Thu Jun  4 16:32:59 2009
From: glen.j.barber at gmail.com (Glen Barber)
Date: Thu Jun  4 16:33:06 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <4A27D38B.6040108@erdgeist.org>
References: <4A27D38B.6040108@erdgeist.org>
Message-ID: <4ad871310906040932n1e78c30do773c8bc92bf547fb@mail.gmail.com>

Hi, Dirk

On Thu, Jun 4, 2009 at 10:00 AM, Dirk Engling <erdgeist@erdgeist.org> wrote:
> Dear fellow hackers,
>
> since jail can be bound on multiple IP addresses I tend to clone
> multiple loopback interfaces and add one loopback address to each jail
>
> cloned_interfaces="lo1 lo2 lo3"
> ifconfig_lo1_alias0="inet 127.0.0.2 netmask 0xffffffff"
> ifconfig_lo2_alias0="inet 127.0.0.3 netmask 0xffffffff"
> ifconfig_lo3_alias0="inet 127.0.0.4 netmask 0xffffffff"
> ..
>
> no this is not yet optimal, since I can not run several jails on a
> single external IP anymore, but at least local daemons are not visible
> to the outside world, anymore.
>

This doesn't answer your _real_ question, but here's a suggestion.

There are a few other ways you could do this with the addressing --
maybe it'll be less confusing for you.  The APIPA address pool
(168.254.x.x/16) is also non-routable.  You could change your aliased
interfaces to use this range, which may clear things up for you, and
the jails will still retain their loopback address.


-- 
Glen Barber
http://www.dev-urandom.com
http://www.linkedin.com/in/glenjbarber
From kientzle at freebsd.org  Thu Jun  4 16:44:05 2009
From: kientzle at freebsd.org (Tim Kientzle)
Date: Thu Jun  4 16:44:12 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
References: <20090604093831.GE48776@hoeg.nl>
	<31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
Message-ID: <4A27F105.4040109@freebsd.org>

Erik Cederstrand wrote:
> 
> LLVM provides a linker (http://llvm.org/cmds/llvm-ld.html) but "it 
> doesn't interact correctly with conventional nm/ar/etc" 
> (http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005296.html). 

In what way does it not interact correctly?

Kai Wang wrote a new libarchive-based ar/nm that's
in -CURRENT; we could possibly augment those to work
with llvm-ld or modify llvm-ld to work with them,
depending on the nature of the disagreement.

Tim

From bzeeb-lists at lists.zabbadoz.net  Thu Jun  4 21:01:19 2009
From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb)
Date: Thu Jun  4 21:01:26 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <4A27D38B.6040108@erdgeist.org>
References: <4A27D38B.6040108@erdgeist.org>
Message-ID: <20090604203905.B12292@maildrop.int.zabbadoz.net>

On Thu, 4 Jun 2009, Dirk Engling wrote:

Hi,

> However, grep -R 127.0.0.1 /etc reveals, that sendmail in many places
> assumes localhost to be on 127.0.0.1 instead of looking it up in
> /etc/hosts or using 127.0.0.0/8 to identify a local connection.

or possibly other methods that would find even more things to be
"local".


> I worry that more programmers made those assumptions, possibly breaking
> more tools.

yes, bind tools are another of those things that have problems with
various address magics.


> My question is: Who's the right guy to beg to fix sendmail or
> alternatively would it be smart to allow each jail to have its own

If programmers assume 127.0.0.1 is hte one and only loopback it's
because of two things - 1) this has been done in the very old days
where people updated the hosts file with uucp to know all hosts in the
nwetwork and was never updated.  or 2) they are clueless or lazy.


> concept of 127.0.0.1 on a dummy interface mapped to all jails, that

As others mentioned connection from/to 127.0.0.1 will be mapped to the
primary address of the jail; if you listen on 127.0.0.1 and the
primary address is a public address you will be visible to the world
(given your base system routes and permits that address to be
reached). But that's been like that since probably 4.0.

With the virtual network stack you will be bale to have your own
loopback with each jail do not even think about doing something like
this; it would never ever hit the tree anymore and it has been done by
others already (for you - and others;).


/bz

-- 
Bjoern A. Zeeb                      The greatest risk is not taking one.
From gshapiro at freebsd.org  Thu Jun  4 21:21:29 2009
From: gshapiro at freebsd.org (Gregory Shapiro)
Date: Thu Jun  4 21:21:35 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <20090604203905.B12292@maildrop.int.zabbadoz.net>
References: <4A27D38B.6040108@erdgeist.org>
	<20090604203905.B12292@maildrop.int.zabbadoz.net>
Message-ID: <20090604212123.GK59532@rugsucker.smi.sendmail.com>

> If programmers assume 127.0.0.1 is hte one and only loopback it's
> because of two things - 1) this has been done in the very old days
> where people updated the hosts file with uucp to know all hosts in the
> nwetwork and was never updated.  or 2) they are clueless or lazy.

To avoid being labeled clueless or lazy, I'll offer a third option.
The software authors have to support a lot of systems with broken setups
that can't actually resolve 'localhost'.  We (the sendmail authors)
originally had localhost in there but we got so many complaints that we
switched it to 127.0.0.1:

8.12.7/8.12.7   2002/12/29
...
        CONFIG: Use FEATURE(`msp', `[127.0.0.1]') in submit.mc by default
                to avoid problems with hostname resolution for localhost
                which on many systems does not resolve to 127.0.0.1 (or
                ::1 for IPv6).  If you do not use IPv4 but only IPv6 then
                you need to change submit.mc accordingly, see the comment
                in the file itself.


Note that it is only in the configuration file (via cf/feature/msp.m4
and cf/m4/proto.m4).  The source code doesn't contain any references
to 127.0.0.1.  In the case of sendmail, you can change both the client
(adjust parameter to FEATURE(msp) in your submit mc file) or server (use
FEATURE(no_default_msp) and your own DAEMON_OPTIONS() definition in your
MTA mc file) to not assume 127.0.0.1, it is just the default.

In my opinion, assuming 127.0.0.1 will work is much safer than assuming
localhost will resolve.  Hopefully that will change at some point, but
for now, sendmail will stick with the safer assumption.
From bzeeb-lists at lists.zabbadoz.net  Thu Jun  4 22:25:08 2009
From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb)
Date: Thu Jun  4 22:25:15 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <20090604212123.GK59532@rugsucker.smi.sendmail.com>
References: <4A27D38B.6040108@erdgeist.org>
	<20090604203905.B12292@maildrop.int.zabbadoz.net>
	<20090604212123.GK59532@rugsucker.smi.sendmail.com>
Message-ID: <20090604220734.H12292@maildrop.int.zabbadoz.net>

On Thu, 4 Jun 2009, Gregory Shapiro wrote:

>> If programmers assume 127.0.0.1 is hte one and only loopback it's
>> because of two things - 1) this has been done in the very old days
>> where people updated the hosts file with uucp to know all hosts in the
>> nwetwork and was never updated.  or 2) they are clueless or lazy.
>
> To avoid being labeled clueless or lazy, I'll offer a third option.

For sendmail I had much rather thought about option 1) and absolutely
not about option 2) as if you have ever known what S5 was and could no
longer read your $ sign and 4 on the keyb you know that sendmail
people are not lazy or clueless!  Back to the OI problem:

> 8.12.7/8.12.7   2002/12/29

yes, that's not from stone age:(


It's quite said, that systems not being able to poperly resolve
hostnames are doing emails these days.

Anyway, yeah, it's good that it's only config files; nethertheless I
don't like it as you probably do not either.   Have you ever thought
about having those files changed just for FreeBSD?  Or had there been
porblems on FreeBSD systems with localhost as well?

Has it been a special time with localhost and IPv6 that casued
problems as 2002 has been rather late in terms of sendmail and resolver
etc.  What had started to cause those problems?

/bz

-- 
Bjoern A. Zeeb                      The greatest risk is not taking one.
From gshapiro at freebsd.org  Thu Jun  4 22:34:52 2009
From: gshapiro at freebsd.org (Gregory Shapiro)
Date: Thu Jun  4 22:34:58 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <20090604220734.H12292@maildrop.int.zabbadoz.net>
References: <4A27D38B.6040108@erdgeist.org>
	<20090604203905.B12292@maildrop.int.zabbadoz.net>
	<20090604212123.GK59532@rugsucker.smi.sendmail.com>
	<20090604220734.H12292@maildrop.int.zabbadoz.net>
Message-ID: <20090604223448.GD465@rugsucker.smi.sendmail.com>

>> 8.12.7/8.12.7   2002/12/29
> yes, that's not from stone age:(

Well, the way things move, it almost is.  :)

> Have you ever thought about having those files changed just for
> FreeBSD?  Or had there been porblems on FreeBSD systems with localhost
> as well?

Now that you mention it, we are already using separate FreeBSD files in
src/etc/sendmail/freebsd{,.submit}.mc so I can do something different
for FreeBSD.  I think we are still in code slush and not freeze for 8.0.
I'll double check and if so, make the change tonight (with a note in
UPDATING).  As I recall, nobody reported a problem resolving localhost
under FreeBSD.

> Has it been a special time with localhost and IPv6 that caused
> problems as 2002 has been rather late in terms of sendmail and
> resolver etc.  What had started to cause those problems?

Sadly, that didn't stick in my memory.  The MTA handles IPv4 fallback
on a failed IPv6 connection if an IPv4 address is available so I don't
think IPv6 enters into it.  If I recall correctly, it was simply a
matter of localhost returning h_errno == HOST_NOT_FOUND or the resolver
returning NXDOMAIN.  I'll check with Claus to see if he remembers.

Another scenario that I still see today (as owner of localhost.org)
is sites that have a bogus 'domain' or 'search' such that the
lookup for 'localhost' returns information about 'localhost.com' or
'localhost.org'.  I've tried to improve that situation by mapping
localhost.org's A record to 127.0.0.1.  Unfortunately, the owners of
localhost.com and localhost.net (as well as possibly everything other
TLD) have not done the same.
From gshapiro at freebsd.org  Thu Jun  4 22:56:31 2009
From: gshapiro at freebsd.org (Gregory Shapiro)
Date: Thu Jun  4 22:56:38 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <20090604223448.GD465@rugsucker.smi.sendmail.com>
References: <4A27D38B.6040108@erdgeist.org>
	<20090604203905.B12292@maildrop.int.zabbadoz.net>
	<20090604212123.GK59532@rugsucker.smi.sendmail.com>
	<20090604220734.H12292@maildrop.int.zabbadoz.net>
	<20090604223448.GD465@rugsucker.smi.sendmail.com>
Message-ID: <20090604225628.GE465@rugsucker.smi.sendmail.com>

> I'll check with Claus to see if he remembers.

Claus still had the mail archive from back then.  It was my scenario.
Specifically, Linux systems with certain versions of glibc would append
the domain/search in /etc/resolv.conf causing localhost lookups to
possibly return offsite systems.  This would of course cause problems
with mail delivery.
From gshapiro at freebsd.org  Fri Jun  5 04:26:21 2009
From: gshapiro at freebsd.org (Gregory Shapiro)
Date: Fri Jun  5 04:26:27 2009
Subject: Jails, loopback interfaces and sendmail
In-Reply-To: <20090604223448.GD465@rugsucker.smi.sendmail.com>
References: <4A27D38B.6040108@erdgeist.org>
	<20090604203905.B12292@maildrop.int.zabbadoz.net>
	<20090604212123.GK59532@rugsucker.smi.sendmail.com>
	<20090604220734.H12292@maildrop.int.zabbadoz.net>
	<20090604223448.GD465@rugsucker.smi.sendmail.com>
Message-ID: <20090605042617.GC639@monkeyboy.local>

> Now that you mention it, we are already using separate FreeBSD files in
> src/etc/sendmail/freebsd{,.submit}.mc so I can do something different
> for FreeBSD.  I think we are still in code slush and not freeze for 8.0.
> I'll double check and if so, make the change tonight (with a note in
> UPDATING).  As I recall, nobody reported a problem resolving localhost
> under FreeBSD.

I'm going to hold off making this change.  It looks like sendmail will
also add the domain in searching since the name isn't qualified:

        /*
        **  Build the search list.
        **      If there is at least one dot in name, start with a null
        **      domain to search the unmodified name first.
        **      If name does not end with a dot and search up local domain
        **      tree desired, append each local domain component to the
        **      search list; if name contains no dots and default domain
        **      name is desired, append default domain name to search list;
        **      else if name ends in a dot, remove that dot.
        */

This leads to lookups that can cause problems:

dns_getcanonname(localhost, trymx=0)
dns_getcanonname: trying localhost.gshapiro.net (AAAA)
        NO: errno=0, h_errno=4
dns_getcanonname: trying localhost.gshapiro.net (A)
        YES
dns_getcanonname: localhost.gshapiro.net
getmxrr([localhost]) returns 1 value(s):
        localhost.gshapiro.net.

We will have to look deeper, perhaps changing the code not to add the
default domain/search list in the case of "localhost".

For now, users who want to use localhost instead of 127.0.0.1 can make
the following change to their /etc/mail/`hostname`.submit.mc:

@@ -22,6 +22,5 @@
 define(`confTIME_ZONE', `USE_TZ')dnl
 define(`confDONT_INIT_GROUPS', `True')dnl
 define(`confBIND_OPTS', `WorkAroundBrokenAAAA')dnl
-dnl
-dnl If you use IPv6 only, change [127.0.0.1] to [IPv6:::1]
-FEATURE(`msp', `[127.0.0.1]')dnl
+DAEMON_OPTIONS(`Name=NoMTA, Addr=localhost, M=E')dnl
+FEATURE(`msp', `[localhost]')dnl

From erik at cederstrand.dk  Fri Jun  5 07:45:47 2009
From: erik at cederstrand.dk (Erik Cederstrand)
Date: Fri Jun  5 11:25:56 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <20090604123801.GA34971@freebsd.org>
References: <20090604093831.GE48776@hoeg.nl>
	<31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
	<20090604123801.GA34971@freebsd.org>
Message-ID: <DF7D22CF-94C1-4B27-A216-E4AD5FEBCA53@cederstrand.dk>

Hi Roman

Den 04/06/2009 kl. 14.38 skrev Roman Divacky:
>
> you could use llvm-ld (see the wiki for instructions how to do it).  
> there's also some
> effort to make gnu ld usable with llvm LTO and I guess the patch  
> could be backported
> to our ld. I guess

As I understand the reply from Eli Friedman[1] this is not possible  
without at least some hacking.

The LTO work in GNU ld[2] is under GPLv3[3], as is gold[4], which  
makes backporting patches a sticky issue.

Erik

[1] http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005296.html
[2] http://gcc.gnu.org/wiki/LinkTimeOptimization
[3] http://gcc.gnu.org/svn/gcc/branches/lto/lto-plugin/lto-plugin.c
[4] http://sourceware.org/cgi-bin/cvsweb.cgi/src/gold/gold.cc?rev=1.63&content-type=text/x-cvsweb-markup&cvsroot=src
From jhb at freebsd.org  Fri Jun  5 13:55:09 2009
From: jhb at freebsd.org (John Baldwin)
Date: Fri Jun  5 13:55:30 2009
Subject: Inline function (difficult  debug)
In-Reply-To: <4A27C5E8.7020902@peterhost.ru>
References: <4A2792B7.5010006@peterhost.ru>
	<200906040749.17206.jhb@freebsd.org>
	<4A27C5E8.7020902@peterhost.ru>
Message-ID: <200906050938.25706.jhb@freebsd.org>

On Thursday 04 June 2009 9:02:32 am Menshikov Konstantin wrote:
> John Baldwin wrote:
> > On Thursday 04 June 2009 5:24:07 am Menshikov Konstantin wrote:
> >   
> >> Hi.
> >> My system FreeBSD 7.1 RELEASE periodically freeze.
> >>
> >> I`m compiling kernel with WITNESS and get backtrace.
> >>
> >> #0 doadump () at pcpu.h:195
> >> #1 0xffffffff801a899c in db_fncall (dummy1=Variable "dummy1" is not 
> >> available.
> >> ) at /usr/src/sys/ddb/db_command.c:516
> >> #2 0xffffffff801a8ecf in db_command (last_cmdp=0xffffffff80825688, 
> >> cmd_table=0x0, dopager=1)
> >> at /usr/src/sys/ddb/db_command.c:413
> >> #3 0xffffffff801a90e0 in db_command_loop () at 
> >> /usr/src/sys/ddb/db_command.c:466
> >> #4 0xffffffff801aaa19 in db_trap (type=Variable "type" is not available.
> >> ) at /usr/src/sys/ddb/db_main.c:228
> >> #5 0xffffffff803d5e55 in kdb_trap (type=3, code=0, 
> >> tf=0xffffffffabf284c0) at /usr/src/sys/kern/subr_kdb.c:524
> >> #6 0xffffffff8059a0e5 in trap (frame=0xffffffffabf284c0) at 
> >> /usr/src/sys/amd64/amd64/trap.c:538
> >> #7 0xffffffff8057f60e in calltrap () at 
> >> /usr/src/sys/amd64/amd64/exception.S:209
> >> #8 0xffffffff803d602d in kdb_enter_why (why=0xffffffff80628137 
> >> "witness", msg=0xa <Address 0xa out of bounds>)
> >> at cpufunc.h:63
> >> #9 0xffffffff803e70c3 in witness_checkorder (lock=Variable "lock" is not 
> >> available.
> >> ) at /usr/src/sys/kern/subr_witness.c:1126
> >> #10 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff8084fad0, 
> >> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
> >> ) at /usr/src/sys/kern/kern_mutex.c:227
> >> #11 0xffffffff80319838 in sc_puts (scp=0xffffffff8084f980, 
> >> buf=0xffffffffabf286a7 "a@\az\200????", len=1)
> >> at /usr/src/sys/dev/syscons/syscons.c:2519
> >> #12 0xffffffff8031b3da in sc_cnputc (cd=Variable "cd" is not available.
> >> ) at /usr/src/sys/dev/syscons/syscons.c:1561
> >> #13 0xffffffff803f9d89 in cnputc (c=97) 
at /usr/src/sys/kern/tty_cons.c:632
> >> #14 0xffffffff803d993b in putchar (c=97, arg=Variable "arg" is not 
> >> available.
> >> ) at /usr/src/sys/kern/subr_prf.c:421
> >> #15 0xffffffff803d7f22 in kvprintf (fmt=0xffffffff80628b89 "cquiring 
> >> duplicate lock of same type: \"%s\"\n",
> >> func=0xffffffff803d98d0 <putchar>, arg=0xffffffffabf28890, radix=10, 
> >> ap=Variable "ap" is not available.
> >> ) at /usr/src/sys/kern/subr_prf.c:674
> >> #16 0xffffffff803d9384 in printf (fmt=Variable "fmt" is not available.
> >> ) at /usr/src/sys/kern/subr_prf.c:314
> >> #17 0xffffffff803e705d in witness_checkorder (lock=0xffffffff808668f8, 
> >> flags=0,
> >> file=0xffffffff80626cf8 "/usr/src/sys/kern/subr_sleepqueue.c", line=232) 
> >> at /usr/src/sys/kern/subr_witness.c:948
> >> #18 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff808668f8, 
> >> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
> >> ) at /usr/src/sys/kern/kern_mutex.c:227
> >> #19 0xffffffff803b2271 in wakeup (ident=0xffffffff80850ac0) at 
> >> /usr/src/sys/kern/kern_synch.c:341
> >> #20 0xffffffff803aed95 in tdsignal (p=0xffffff00017978f0, 
> >> td=0xffffff000146a6e0, sig=14, ksi=Variable "ksi" is not available.
> >> )
> >> at /usr/src/sys/kern/kern_sig.c:2292
> >> #21 0xffffffff803ba2de in realitexpire (arg=Variable "arg" is not 
available.
> >> ) at /usr/src/sys/kern/kern_time.c:669
> >> #22 0xffffffff803bbe1a in softclock (dummy=Variable "dummy" is not 
> >> available.
> >> ) at /usr/src/sys/kern/kern_timeout.c:274
> >> #23 0xffffffff8038c120 in ithread_loop (arg=0xffffff00010ebc00) at 
> >> /usr/src/sys/kern/kern_intr.c:1088
> >> #24 0xffffffff8038978a in fork_exit (callout=0xffffffff8038c040 
> >> <ithread_loop>, arg=0xffffff00010ebc00,
> >> frame=0xffffffffabf28c80) at /usr/src/sys/kern/kern_fork.c:804
> >> #25 0xffffffff8057f9de in fork_trampoline () at 
> >> /usr/src/sys/amd64/amd64/exception.S:455
> >>
> >> But file /usr/src/sys/kern/kern_sig.c:2292
> >> hasn`t run function wake up on string 2292
> >> I`m think, what compiler use inline function, therefore code and trace 
> >> differ.
> >> Tell me please, how compile kernel without any inline function.
> >>     
> >
> > makeoptions DEBUG="-g -fno-inline"
> >
> > However. I think you already have enough information to debug this.  Can 
you 
> > go to frame 17 and do an 'l'?  I think the two things you want to print 
out 
> > are "i", "*lock", "*lock1", and "*lock2".
> >
> >   
> Unfortunately, the kernel is not compiled with an option-fno-inline.
> cc -c -O2 -frename-registers -pipe -fno-strict-aliasing  -std=c99 -g 
> -fno-inline -Wall -Wredundant-decls -Wnested-externs 
> -Wstrict-prototypes  -Wmissing-prototypes -Wpointer-arith -Winline 
> -Wcast-qual  -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc  
> -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL 
> -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common 
> -finline-limit=8000 --param inline-unit-growth=100 --param 
> large-function-growth=1000  -fno-omit-frame-pointer -mcmodel=kernel 
> -mno-red-zone  -mfpmath=387 -mno-sse -mno-sse2 -mno-mmx -mno-3dnow  
> -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -Werror  
> /usr/src/sys/cam/cam_periph.c
> cc1: warnings being treated as errors
> /usr/src/sys/cam/cam_periph.c: In function 'scsi_extract_sense':
> /usr/src/sys/cam/scsi/scsi_all.h:1215: warning: function 
> 'scsi_extract_sense' can never be inlined because it is suppressed using 
> -fno-inline
> 
> I`m try build with -O0 options.
> 
> lock in frame 17
> (kgdb) print {struct mtx} 0xffffffff808668f8
> $4 = {lock_object = {lo_name = 0xffffffff806270eb "sleepq chain", 
> lo_type = 0xffffffff806270eb "sleepq chain",
>     lo_flags = 720896, lo_witness_data = {lod_list = {stqe_next = 
> 0xffffffff808793e0}, lod_witness = 0xffffffff808793e0}},
>   mtx_lock = 4, mtx_recurse = 0}
> next in list
> (kgdb) print {struct mtx} 0xffffffff808793e0
> $5 = {lock_object = {lo_name = 0xffffffff806270eb "sleepq chain", 
> lo_type = 0xffffffff807e8860 "\031\006b\200????\n",
>     lo_flags = 2156368928, lo_witness_data = {lod_list = {stqe_next = 
> 0xffffffff80879420},
>       lod_witness = 0xffffffff80879420}}, mtx_lock = 
> 18446744071571216096, mtx_recurse = 2153913763}
> (kgdb) print {struct mtx} 0xffffffff80879420
> next
> $6 = {lock_object = {lo_name = 0xffffffff806205df "process slock", 
> lo_type = 0xffffffff807e8860 "\031\006b\200????\n",
>     lo_flags = 2156368992, lo_witness_data = {lod_list = {stqe_next = 
> 0xffffffff80879460},
>       lod_witness = 0xffffffff80879460}}, mtx_lock = 
> 18446744071571216160, mtx_recurse = 2153889520}

Where did you get the pointer values from?  I meant doing 'frame 17', 'p 
i', 'p *lock', 'p *lock1', 'p *lock2'

-- 
John Baldwin
From kostjn at peterhost.ru  Fri Jun  5 14:58:15 2009
From: kostjn at peterhost.ru (Menshikov Konstantin)
Date: Fri Jun  5 14:58:23 2009
Subject: Inline function (difficult  debug)
In-Reply-To: <200906050938.25706.jhb@freebsd.org>
References: <4A2792B7.5010006@peterhost.ru>
	<200906040749.17206.jhb@freebsd.org>
	<4A27C5E8.7020902@peterhost.ru>
	<200906050938.25706.jhb@freebsd.org>
Message-ID: <4A293302.5040306@peterhost.ru>

John Baldwin wrote:
> On Thursday 04 June 2009 9:02:32 am Menshikov Konstantin wrote:
>   
>> John Baldwin wrote:
>>     
>>> On Thursday 04 June 2009 5:24:07 am Menshikov Konstantin wrote:
>>>   
>>>       
>>>> Hi.
>>>> My system FreeBSD 7.1 RELEASE periodically freeze.
>>>>
>>>> I`m compiling kernel with WITNESS and get backtrace.
>>>>
>>>> #0 doadump () at pcpu.h:195
>>>> #1 0xffffffff801a899c in db_fncall (dummy1=Variable "dummy1" is not 
>>>> available.
>>>> ) at /usr/src/sys/ddb/db_command.c:516
>>>> #2 0xffffffff801a8ecf in db_command (last_cmdp=0xffffffff80825688, 
>>>> cmd_table=0x0, dopager=1)
>>>> at /usr/src/sys/ddb/db_command.c:413
>>>> #3 0xffffffff801a90e0 in db_command_loop () at 
>>>> /usr/src/sys/ddb/db_command.c:466
>>>> #4 0xffffffff801aaa19 in db_trap (type=Variable "type" is not available.
>>>> ) at /usr/src/sys/ddb/db_main.c:228
>>>> #5 0xffffffff803d5e55 in kdb_trap (type=3, code=0, 
>>>> tf=0xffffffffabf284c0) at /usr/src/sys/kern/subr_kdb.c:524
>>>> #6 0xffffffff8059a0e5 in trap (frame=0xffffffffabf284c0) at 
>>>> /usr/src/sys/amd64/amd64/trap.c:538
>>>> #7 0xffffffff8057f60e in calltrap () at 
>>>> /usr/src/sys/amd64/amd64/exception.S:209
>>>> #8 0xffffffff803d602d in kdb_enter_why (why=0xffffffff80628137 
>>>> "witness", msg=0xa <Address 0xa out of bounds>)
>>>> at cpufunc.h:63
>>>> #9 0xffffffff803e70c3 in witness_checkorder (lock=Variable "lock" is not 
>>>> available.
>>>> ) at /usr/src/sys/kern/subr_witness.c:1126
>>>> #10 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff8084fad0, 
>>>> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
>>>> ) at /usr/src/sys/kern/kern_mutex.c:227
>>>> #11 0xffffffff80319838 in sc_puts (scp=0xffffffff8084f980, 
>>>> buf=0xffffffffabf286a7 "a@\az\200????", len=1)
>>>> at /usr/src/sys/dev/syscons/syscons.c:2519
>>>> #12 0xffffffff8031b3da in sc_cnputc (cd=Variable "cd" is not available.
>>>> ) at /usr/src/sys/dev/syscons/syscons.c:1561
>>>> #13 0xffffffff803f9d89 in cnputc (c=97) 
>>>>         
> at /usr/src/sys/kern/tty_cons.c:632
>   
>>>> #14 0xffffffff803d993b in putchar (c=97, arg=Variable "arg" is not 
>>>> available.
>>>> ) at /usr/src/sys/kern/subr_prf.c:421
>>>> #15 0xffffffff803d7f22 in kvprintf (fmt=0xffffffff80628b89 "cquiring 
>>>> duplicate lock of same type: \"%s\"\n",
>>>> func=0xffffffff803d98d0 <putchar>, arg=0xffffffffabf28890, radix=10, 
>>>> ap=Variable "ap" is not available.
>>>> ) at /usr/src/sys/kern/subr_prf.c:674
>>>> #16 0xffffffff803d9384 in printf (fmt=Variable "fmt" is not available.
>>>> ) at /usr/src/sys/kern/subr_prf.c:314
>>>> #17 0xffffffff803e705d in witness_checkorder (lock=0xffffffff808668f8, 
>>>> flags=0,
>>>> file=0xffffffff80626cf8 "/usr/src/sys/kern/subr_sleepqueue.c", line=232) 
>>>> at /usr/src/sys/kern/subr_witness.c:948
>>>> #18 0xffffffff8039e341 in _mtx_lock_spin_flags (m=0xffffffff808668f8, 
>>>> opts=0, file=dwarf2_read_address: Corrupted DWARF expression.
>>>> ) at /usr/src/sys/kern/kern_mutex.c:227
>>>> #19 0xffffffff803b2271 in wakeup (ident=0xffffffff80850ac0) at 
>>>> /usr/src/sys/kern/kern_synch.c:341
>>>> #20 0xffffffff803aed95 in tdsignal (p=0xffffff00017978f0, 
>>>> td=0xffffff000146a6e0, sig=14, ksi=Variable "ksi" is not available.
>>>> )
>>>> at /usr/src/sys/kern/kern_sig.c:2292
>>>> #21 0xffffffff803ba2de in realitexpire (arg=Variable "arg" is not 
>>>>         
> available.
>   
>>>> ) at /usr/src/sys/kern/kern_time.c:669
>>>> #22 0xffffffff803bbe1a in softclock (dummy=Variable "dummy" is not 
>>>> available.
>>>> ) at /usr/src/sys/kern/kern_timeout.c:274
>>>> #23 0xffffffff8038c120 in ithread_loop (arg=0xffffff00010ebc00) at 
>>>> /usr/src/sys/kern/kern_intr.c:1088
>>>> #24 0xffffffff8038978a in fork_exit (callout=0xffffffff8038c040 
>>>> <ithread_loop>, arg=0xffffff00010ebc00,
>>>> frame=0xffffffffabf28c80) at /usr/src/sys/kern/kern_fork.c:804
>>>> #25 0xffffffff8057f9de in fork_trampoline () at 
>>>> /usr/src/sys/amd64/amd64/exception.S:455
>>>>
>>>> But file /usr/src/sys/kern/kern_sig.c:2292
>>>> hasn`t run function wake up on string 2292
>>>> I`m think, what compiler use inline function, therefore code and trace 
>>>> differ.
>>>> Tell me please, how compile kernel without any inline function.
>>>>     
>
>
> Where did you get the pointer values from?  I meant doing 'frame 17', 'p 
> i', 'p *lock', 'p *lock1', 'p *lock2'
>
>   

Excuse I has not understood you.
Disable optimization -O0, disable inline function.
I`m get new backtrace
#0  doadump () at /usr/src/sys/kern/kern_shutdown.c:244
#1  0xffffffff801c58fc in db_fncall_generic (addr=-2142879744, 
rv=0xffffffffac0f9d88, nargs=0, args=0xffffffffac0f9d90)
    at /usr/src/sys/ddb/db_command.c:516
#2  0xffffffff801c5805 in db_fncall (dummy1=-2136396608, dummy2=0, 
dummy3=-1408262592,
    dummy4=0xffffffffac0f9e30 "`\224\224\200????????\r") at 
/usr/src/sys/ddb/db_command.c:568
#3  0xffffffff801c5546 in db_command (last_cmdp=0xffffffff809d14c0, 
cmd_table=0x0, dopager=1)
    at /usr/src/sys/ddb/db_command.c:413
#4  0xffffffff801c5644 in db_command_loop () at 
/usr/src/sys/ddb/db_command.c:466
#5  0xffffffff801c81e9 in db_trap (type=3, code=0) at 
/usr/src/sys/ddb/db_main.c:228
#6  0xffffffff8049d629 in kdb_trap (type=3, code=0, 
tf=0xffffffffac0fa0f0) at /usr/src/sys/kern/subr_kdb.c:524
#7  0xffffffff807180d2 in trap (frame=0xffffffffac0fa0f0) at 
/usr/src/sys/amd64/amd64/trap.c:538
#8  0xffffffff806f7cee in calltrap () at 
/usr/src/sys/amd64/amd64/exception.S:209
#9  0xffffffff8049d0c5 in breakpoint () at cpufunc.h:63
#10 0xffffffff8049d0b2 in kdb_enter_why (why=0xffffffff807c9140 
"witness", msg=0xffffffff807c97f0 "witness_checkorder")
    at /usr/src/sys/kern/subr_kdb.c:315
#11 0xffffffff804b3db8 in witness_checkorder (lock=0xffffffff809fb8b0, 
flags=9,
    file=0xffffffff807aeb30 "/usr/src/sys/dev/syscons/syscons.c", 
line=2519) at /usr/src/sys/kern/subr_witness.c:1126
#12 0xffffffff80453ead in _mtx_lock_spin_flags (m=0xffffffff809fb8b0, 
opts=0,
    file=0xffffffff807aeb30 "/usr/src/sys/dev/syscons/syscons.c", 
line=2519) at /usr/src/sys/kern/kern_mutex.c:227
#13 0xffffffff803a0d4d in sc_puts (scp=0xffffffff809fb760, 
buf=0xffffffffac0fa337 "a`?\237\200????\200?)\001", len=1)
    at /usr/src/sys/dev/syscons/syscons.c:2519
#14 0xffffffff8039f063 in sc_cnputc (cd=0xffffffff80949460, c=97) at 
/usr/src/sys/dev/syscons/syscons.c:1561
#15 0xffffffff804cf82f in cnputc (c=97) at /usr/src/sys/kern/tty_cons.c:632
#16 0xffffffff804a0864 in putcons (c=97, ap=0xffffffffac0fa6b0) at 
/usr/src/sys/kern/subr_prf.c:372
#17 0xffffffff804a09a4 in putchar (c=97, arg=0xffffffffac0fa6b0) at 
/usr/src/sys/kern/subr_prf.c:413
#18 0xffffffff804a0ee9 in kvprintf (fmt=0xffffffff807c9911 "cquiring 
duplicate lock of same type: \"%s\"\n",
    func=0xffffffff804a0920 <putchar>, arg=0xffffffffac0fa6b0, radix=10, 
ap=0xffffffffac0fa6e0)
    at /usr/src/sys/kern/subr_prf.c:599
#19 0xffffffff804a07a4 in printf (fmt=0xffffffff807c9910 "acquiring 
duplicate lock of same type: \"%s\"\n")
    at /usr/src/sys/kern/subr_prf.c:314
#20 0xffffffff804b374e in witness_checkorder (lock=0xffffffff80a13448, 
flags=9,
    file=0xffffffff807c7a38 "/usr/src/sys/kern/subr_sleepqueue.c", 
line=232) at /usr/src/sys/kern/subr_witness.c:948
#21 0xffffffff80453ead in _mtx_lock_spin_flags (m=0xffffffff80a13448, 
opts=0,
    file=0xffffffff807c7a38 "/usr/src/sys/kern/subr_sleepqueue.c", 
line=232) at /usr/src/sys/kern/kern_mutex.c:227
#22 0xffffffff804a8472 in sleepq_lock (wchan=0xffffffff809fc8a0) at 
/usr/src/sys/kern/subr_sleepqueue.c:232
#23 0xffffffff80470175 in wakeup (ident=0xffffffff809fc8a0) at 
/usr/src/sys/kern/kern_synch.c:341
#24 0xffffffff806cf110 in kick_proc0 () at /usr/src/sys/vm/vm_glue.c:761
#25 0xffffffff8046aa16 in tdsigwakeup (td=0xffffff000172a000, sig=14, 
action=0x2, intrval=4)
    at /usr/src/sys/kern/kern_sig.c:2422
#26 0xffffffff8046a5d1 in do_tdsignal (p=0xffffff0001727000, 
td=0xffffff000172a000, sig=14, ksi=0xffffff00017dfd68)
    at /usr/src/sys/kern/kern_sig.c:2292
#27 0xffffffff80469d48 in tdsignal (p=0xffffff0001727000, td=0x0, 
sig=14, ksi=0xffffff00017dfd68)
    at /usr/src/sys/kern/kern_sig.c:2029
#28 0xffffffff80469cd9 in psignal_event (p=0xffffff0001727000, 
sigev=0xffffff00017dfcd8, ksi=0xffffff00017dfd68)
    at /usr/src/sys/kern/kern_sig.c:2017
#29 0xffffffff8047c036 in itimer_fire (it=0xffffff00017dfca8) at 
/usr/src/sys/kern/kern_time.c:1419
#30 0xffffffff8047be84 in realtimer_expire (arg=0xffffff00017dfca8) at 
/usr/src/sys/kern/kern_time.c:1396
#31 0xffffffff8047c9a6 in softclock (dummy=0x0) at 
/usr/src/sys/kern/kern_timeout.c:274
#32 0xffffffff8043c057 in ithread_execute_handlers 
(p=0xffffff0001105000, ie=0xffffff000110c000)
---Type <return> to continue, or q <return> to quit---
    at /usr/src/sys/kern/kern_intr.c:1088
#33 0xffffffff8043c28f in ithread_loop (arg=0xffffff00010ecbe0) at 
/usr/src/sys/kern/kern_intr.c:1175
#34 0xffffffff8043a3d7 in fork_exit (callout=0xffffffff8043c1d0 
<ithread_loop>, arg=0xffffff00010ecbe0,
    frame=0xffffffffac0fac80) at /usr/src/sys/kern/kern_fork.c:804
#35 0xffffffff806f80be in fork_trampoline () at 
/usr/src/sys/amd64/amd64/exception.S:455


Frame 20
(kgdb) f 20
#20 0xffffffff804b374e in witness_checkorder (lock=0xffffffff80a13448, 
flags=9,
    file=0xffffffff807c7a38 "/usr/src/sys/kern/subr_sleepqueue.c", 
line=232) at /usr/src/sys/kern/subr_witness.c:948
948            printf("acquiring duplicate lock of same type: \"%s\"\n",
(kgdb) p *lock
$1 = {lo_name = 0xffffffff807c7a20 "sleepq chain", lo_type = 
0xffffffff807c7a20 "sleepq chain", lo_flags = 720896,
  lo_witness_data = {lod_list = {stqe_next = 0xffffffff80a251a0}, 
lod_witness = 0xffffffff80a251a0}}
(kgdb) p *lock1
$2 = {li_lock = 0xffffffff80a12db8, li_file = 0xffffffff807c22f5 
"/usr/src/sys/kern/kern_sig.c", li_line = 2291,
  li_flags = 65536}
(kgdb) p *lock2
$3 = {li_lock = 0xffffffff80a10d40, li_file = 0xffffff0001105000 
"xT\020\001", li_line = 0, li_flags = 0}
(kgdb) p i
$4 = -2139345904


Frame 11
(kgdb) f 11
#11 0xffffffff804b3db8 in witness_checkorder (lock=0xffffffff809fb8b0, 
flags=9,
    file=0xffffffff807aeb30 "/usr/src/sys/dev/syscons/syscons.c", 
line=2519) at /usr/src/sys/kern/subr_witness.c:1126
1126            kdb_enter_why(KDB_WHY_WITNESS, __func__);
(kgdb) p *lock
$6 = {lo_name = 0xffffffff807aebc5 "scrlock", lo_type = 
0xffffffff807aebc5 "scrlock", lo_flags = 8585216,
  lo_witness_data = {lod_list = {stqe_next = 0xffffffff80a25320}, 
lod_witness = 0xffffffff80a25320}}
(kgdb) p *lock1
$7 = {li_lock = 0xffffffff80a12db8, li_file = 0xffffffff807c22f5 
"/usr/src/sys/kern/kern_sig.c", li_line = 2291,
  li_flags = 65536}
(kgdb) p *lock2
$8 = {li_lock = 0xffffff0001727030, li_file = 0xffffffff807c22f5 
"/usr/src/sys/kern/kern_sig.c", li_line = 2176,
  li_flags = 65536}
(kgdb) p i
$9 = -1

Thanks.
From james at mansionfamily.plus.com  Fri Jun  5 22:55:40 2009
From: james at mansionfamily.plus.com (James Mansion)
Date: Fri Jun  5 22:55:48 2009
Subject: pxeboot and http
Message-ID: <4A299C05.1030509@mansionfamily.plus.com>

Is the ability for pxeboot to load from an HTTP source merged - or due 
to be merged?

It was a gsoc from 2007 wasn't it?

James



From erik at cederstrand.dk  Sat Jun  6 19:28:02 2009
From: erik at cederstrand.dk (Erik Cederstrand)
Date: Sat Jun  6 19:28:09 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <4A27F105.4040109@freebsd.org>
References: <20090604093831.GE48776@hoeg.nl>
	<31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
	<4A27F105.4040109@freebsd.org>
Message-ID: <686A733E-2F33-4C40-A516-7D3A8E5E431E@cederstrand.dk>


Den 04/06/2009 kl. 18.06 skrev Tim Kientzle:

> Erik Cederstrand wrote:
>> LLVM provides a linker (http://llvm.org/cmds/llvm-ld.html) but "it  
>> doesn't interact correctly with conventional nm/ar/etc" (http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005296.html 
>> ).
>
> In what way does it not interact correctly?

There was not much more help to get from the Clang list,  
unfortunately. The code lives at http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-ld/ 
  and looks very well-documented and structured. Unfortunately, it is  
a bit over my head to see what needs to be fixed and actually fix it.

Thanks,
Erik
From ivoras at freebsd.org  Sat Jun  6 19:44:51 2009
From: ivoras at freebsd.org (Ivan Voras)
Date: Sat Jun  6 19:45:00 2009
Subject: SoC 2009: puffs port for FreeBSD
In-Reply-To: <a096b1430906030548w389cebefy6a9553ba07ff11f3@mail.gmail.com>
References: <a096b1430906030548w389cebefy6a9553ba07ff11f3@mail.gmail.com>
Message-ID: <h0egv6$r4u$1@ger.gmane.org>

Tatsiana Severyna wrote:
> Hi,
> 
> I'm Tatsiana Severyna from Belarus. I'll be working on porting puffs
> from NetBSD as a part of this year's summer of code program. refuse
> library (fuse compatibility layer on top of puffs) will also be ported
> to FreeBSD as during the summer.  My mentor is Konstantin Belousov.

This is very interesting! I hope it will go well!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090606/27b9e20f/signature.pgp
From ivoras at freebsd.org  Sat Jun  6 19:54:53 2009
From: ivoras at freebsd.org (Ivan Voras)
Date: Sat Jun  6 19:55:01 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network>	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
Message-ID: <h0ehhv$sic$1@ger.gmane.org>

Sorry to come into the discussion late, but I just want to confirm
something.

The configuration below is a stripe of four components, each of which is
RAIDZ2, right?

If, as was discussed later in the thread, RAIDZ(2) is more similar to
RAID3 than RAID5 for random performance, the given configuration can be
(very roughly, in the non-sequential access case) expected to deliver
performance of four drives in a RAID0 array?

krad wrote:

> zpool create -O compression=lzjb -O atime=off data raidz2 c3t0d0 c4t0d0
> c8t0d0 c10t0d0 c11t0d0 c3t1d0 c4t1d0 c8t1d0 c9t1d0 c10t1d0 c11t1d0 raidz2
> c3t2d0 c4t2d0 c8t2d0 c9t2d0 c11t2d0 c3t3d0 c4t3d0 c8t3d0 c9t3d0 c10t3d0
> c11t3d0 raidz2 c3t4d0 c4t4d0 c8t4d0 c10t4d0 c11t4d0 c3t5d0 c4t5d0 c8t5d0
> c9t5d0 c10t5d0 c11t5d0 raidz2 c3t6d0 c4t6d0 c8t6d0 c9t6d0 c10t6d0 c11t6d0
> c3t7d0 c4t7d0 c9t7d0 c10t7d0 c11t7d0 spare c10t2d0 c8t7d0

>         NAME         STATE     READ WRITE CKSUM
>         archive-2    ONLINE       0     0     0
>           raidz2     ONLINE       0     0     0
>             c3t0d0   ONLINE       0     0     0
>             c4t0d0   ONLINE       0     0     0
>             c8t0d0   ONLINE       0     0     0
>             c10t0d0  ONLINE       0     0     0
>             c11t0d0  ONLINE       0     0     0
>             c3t1d0   ONLINE       0     0     0
>             c4t1d0   ONLINE       0     0     0
>             c8t1d0   ONLINE       0     0     0
>             c9t1d0   ONLINE       0     0     0
>             c10t1d0  ONLINE       0     0     0
>             c11t1d0  ONLINE       0     0     0
>           raidz2     ONLINE       0     0     0
>             c3t2d0   ONLINE       0     0     0
>             c4t2d0   ONLINE       0     0     0
>             c8t2d0   ONLINE       0     0     0
>             c9t2d0   ONLINE       0     0     0
>             c11t2d0  ONLINE       0     0     0
>             c3t3d0   ONLINE       0     0     0
>             c4t3d0   ONLINE       0     0     0
>             c8t3d0   ONLINE       0     0     0
>             c9t3d0   ONLINE       0     0     0
>             c10t3d0  ONLINE       0     0     0
>             c11t3d0  ONLINE       0     0     0
>           raidz2     ONLINE       0     0     0
>             c3t4d0   ONLINE       0     0     0
>             c4t4d0   ONLINE       0     0     0
>             c8t4d0   ONLINE       0     0     0
>             c10t4d0  ONLINE       0     0     0
>             c11t4d0  ONLINE       0     0     0
>             c3t5d0   ONLINE       0     0     0
>             c4t5d0   ONLINE       0     0     0
>             c8t5d0   ONLINE       0     0     0
>             c9t5d0   ONLINE       0     0     0
>             c10t5d0  ONLINE       0     0     0
>             c11t5d0  ONLINE       0     0     0
>           raidz2     ONLINE       0     0     0
>             c3t6d0   ONLINE       0     0     0
>             c4t6d0   ONLINE       0     0     0
>             c8t6d0   ONLINE       0     0     0
>             c9t6d0   ONLINE       0     0     0
>             c10t6d0  ONLINE       0     0     0
>             c11t6d0  ONLINE       0     0     0
>             c3t7d0   ONLINE       0     0     0
>             c4t7d0   ONLINE       0     0     0
>             c9t7d0   ONLINE       0     0     0
>             c10t7d0  ONLINE       0     0     0
>             c11t7d0  ONLINE       0     0     0
>         spares
>           c10t2d0    AVAIL   
>           c8t7d0     AVAIL   
> 
> errors: No known data errors


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090606/f092cbcf/signature.pgp
From fjwcash at gmail.com  Sat Jun  6 20:16:34 2009
From: fjwcash at gmail.com (Freddie Cash)
Date: Sat Jun  6 20:16:41 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <h0ehhv$sic$1@ger.gmane.org>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<h0ehhv$sic$1@ger.gmane.org>
Message-ID: <b269bc570906061316g37290b5q910da0d3ec266c98@mail.gmail.com>

On Sat, Jun 6, 2009 at 12:54 PM, Ivan Voras<ivoras@freebsd.org> wrote:
> Sorry to come into the discussion late, but I just want to confirm
> something.
>
> The configuration below is a stripe of four components, each of which is
> RAIDZ2, right?
>
> If, as was discussed later in the thread, RAIDZ(2) is more similar to
> RAID3 than RAID5 for random performance, the given configuration can be
> (very roughly, in the non-sequential access case) expected to deliver
> performance of four drives in a RAID0 array?

According to all the Sun documentation, the I/O throughput of a raidz
configuration is equal to that of a single drive.

Hence their recommendation to not use more than 8 or 9 drives in a
single raidz vdev, and to use multiple raidz vdevs.  As you add vdevs,
the throughput increases.

We made the mistake early on of creating a 24-drive raidz2 vdev.
Performance was not very good.  And when we had to replace a drive, it
spent over a week trying to resilver.  But the resilver operation has
to touch every single drive in the raidz vdev.  :(

We remade the pool using 3x 8-drive raidz2 vdevs, and performance has
been great (400 MBytes/s write, almost 3 GBytes/s sequential read, 800
MBytes/s random read).

-- 
Freddie Cash
fjwcash@gmail.com
From kraduk at googlemail.com  Sun Jun  7 00:01:20 2009
From: kraduk at googlemail.com (krad)
Date: Sun Jun  7 01:58:10 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <b269bc570906061316g37290b5q910da0d3ec266c98@mail.gmail.com>
References: <20090530175239.GA25604@logik.internal.network><20090530144354.2255f722@bhuda.mired.org><20090530191840.GA68514@logik.internal.network><20090530162744.5d77e9d1@bhuda.mired.org><A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl><h0ehhv$sic$1@ger.gmane.org>
	<b269bc570906061316g37290b5q910da0d3ec266c98@mail.gmail.com>
Message-ID: <C3E617A51315419EADE74238D68B81ED@uk.tiscali.intl>


>We remade the pool using 3x 8-drive raidz2 vdevs, and performance has
>been great (400 MBytes/s write, almost 3 GBytes/s sequential read, 800
>MBytes/s random read).

Yep that corresponds with what we saw, although we were getting a little
higher write rates with our 46 drive configuration

From kaiwang27 at gmail.com  Sun Jun  7 05:57:31 2009
From: kaiwang27 at gmail.com (Kai Wang)
Date: Sun Jun  7 05:57:43 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <4A27F105.4040109@freebsd.org>
References: <20090604093831.GE48776@hoeg.nl>
	<31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
	<4A27F105.4040109@freebsd.org>
Message-ID: <20090607052555.GA2154@viskning>

On Thu, Jun 04, 2009 at 09:06:29AM -0700, Tim Kientzle wrote:
> Erik Cederstrand wrote:
> > 
> > LLVM provides a linker (http://llvm.org/cmds/llvm-ld.html) but "it 
> > doesn't interact correctly with conventional nm/ar/etc" 
> > (http://lists.cs.uiuc.edu/pipermail/cfe-dev/2009-June/005296.html). 
> 
> In what way does it not interact correctly?

llvm-ar manual page:
	http://llvm.org/cmds/llvm-ar.html

I think the major difference is that llvm-ar is aimed for bitcode (.bc)
files, same as other llvm-xxx tools. (for example, llvm-ld)

Other that that, llvm-ar uses a different symbol table, a different
compression solution. (llvm-ar compresses each member separately, not
the entire archive, this is probably better wrt random access of
members?).  llvm-ar also has a handy -R option which we should
probably add to our ar(1).

And from what I read in the wiki, it looks like llvm-ld can be used as
long as --emit-llvm is specified when compiling?

-Kai
From wojtek at wojtek.tensor.gdynia.pl  Sun Jun  7 07:01:42 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Sun Jun  7 07:01:49 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <b269bc570906061316g37290b5q910da0d3ec266c98@mail.gmail.com>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<h0ehhv$sic$1@ger.gmane.org>
	<b269bc570906061316g37290b5q910da0d3ec266c98@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906070858210.97807@wojtek.tensor.gdynia.pl>

>> (very roughly, in the non-sequential access case) expected to deliver
>> performance of four drives in a RAID0 array?
>
> According to all the Sun documentation, the I/O throughput of a raidz
> configuration is equal to that of a single drive.

exactly what i say. it's like RAID3. Not RAID5 which have close to n times 
single drive throughput on read and rougly n/4 on writes.

> We remade the pool using 3x 8-drive raidz2 vdevs, and performance has
> been great (400 MBytes/s write, almost 3 GBytes/s sequential read, 800

why write performance is so slow? in Sun theory it should have the same 
speed as reads. I would say that it should be even better a bit - 
filesystem get data first in cache and can plan ahead.

> MBytes/s random read).

random read on how big chunks?

Are you sure you get 3GB/s on read? it would mean each drive must be able 
to do 140MB/s

What disks do you use?
From hdante at gmail.com  Sun Jun  7 13:46:08 2009
From: hdante at gmail.com (Henrique Almeida)
Date: Sun Jun  7 14:47:37 2009
Subject: New "libc" project libposix
Message-ID: <c23df7cb0906070620p734f78d9gc68084ccfeff85b1@mail.gmail.com>

 Hello,

 I've created a new project to build a compliant POSIX 2008 library. The
goals of the project are to build a cross platform, easy to use library that
will unify the POSIX implementation on all systems.

 Everybody is welcome, specially "libc" developers. ;-)

 http://libposix.sourceforge.net/

-- 
Henrique Dante de Almeida
hdante@gmail.com
From ivoras at freebsd.org  Mon Jun  8 00:49:53 2009
From: ivoras at freebsd.org (Ivan Voras)
Date: Mon Jun  8 00:49:59 2009
Subject: New "libc" project libposix
In-Reply-To: <c23df7cb0906070620p734f78d9gc68084ccfeff85b1@mail.gmail.com>
References: <c23df7cb0906070620p734f78d9gc68084ccfeff85b1@mail.gmail.com>
Message-ID: <h0hn71$qi0$1@ger.gmane.org>

Henrique Almeida wrote:
>  Hello,
> 
>  I've created a new project to build a compliant POSIX 2008 library. The
> goals of the project are to build a cross platform, easy to use library that
> will unify the POSIX implementation on all systems.
> 
>  Everybody is welcome, specially "libc" developers. ;-)
> 
>  http://libposix.sourceforge.net/

I think that a lot of compliance depends on the kernel - you can't solve
those in libc.

Another thing: are you starting from scratch or using an existing libc?
I imagine that using e.g. FreeBSD's libc and fixing compatiblity issues
will make your whole project complete significantly faster.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090608/b74931bc/signature.pgp
From jhb at freebsd.org  Mon Jun  8 15:33:50 2009
From: jhb at freebsd.org (John Baldwin)
Date: Mon Jun  8 15:34:10 2009
Subject: Inline function (difficult  debug)
In-Reply-To: <4A293302.5040306@peterhost.ru>
References: <4A2792B7.5010006@peterhost.ru>
	<200906050938.25706.jhb@freebsd.org>
	<4A293302.5040306@peterhost.ru>
Message-ID: <200906081031.32271.jhb@freebsd.org>

On Friday 05 June 2009 11:00:18 am Menshikov Konstantin wrote:
> Frame 11
> (kgdb) f 11
> #11 0xffffffff804b3db8 in witness_checkorder (lock=0xffffffff809fb8b0, 
> flags=9,
>     file=0xffffffff807aeb30 "/usr/src/sys/dev/syscons/syscons.c", 
> line=2519) at /usr/src/sys/kern/subr_witness.c:1126
> 1126            kdb_enter_why(KDB_WHY_WITNESS, __func__);
> (kgdb) p *lock
> $6 = {lo_name = 0xffffffff807aebc5 "scrlock", lo_type = 
> 0xffffffff807aebc5 "scrlock", lo_flags = 8585216,
>   lo_witness_data = {lod_list = {stqe_next = 0xffffffff80a25320}, 
> lod_witness = 0xffffffff80a25320}}
> (kgdb) p *lock1
> $7 = {li_lock = 0xffffffff80a12db8, li_file = 0xffffffff807c22f5 
> "/usr/src/sys/kern/kern_sig.c", li_line = 2291,
>   li_flags = 65536}

Ok, can you 'p *lock1->li_lock'?

-- 
John Baldwin
From imp at bsdimp.com  Mon Jun  8 18:07:13 2009
From: imp at bsdimp.com (M. Warner Losh)
Date: Mon Jun  8 18:07:25 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <20090604093831.GE48776@hoeg.nl>
References: <20090604093831.GE48776@hoeg.nl>
Message-ID: <20090608.120552.756910862.imp@bsdimp.com>

In message: <20090604093831.GE48776@hoeg.nl>
            Ed Schouten <ed@80386.nl> writes:
: Good news everyone!
...
: So far we've only done testing on amd64 and i386. A lot of ports are
: probably still broken. Caveat emptor. Beware of dog. Slippery when wet.

"objects in mirror may be larger than they appear"

Do you have size or run-time performance comparisons yet?

Warner
From fjwcash at gmail.com  Mon Jun  8 19:11:19 2009
From: fjwcash at gmail.com (Freddie Cash)
Date: Mon Jun  8 19:11:26 2009
Subject: Request for opinions - gvinum or ccd?
In-Reply-To: <alpine.BSF.2.00.0906070858210.97807@wojtek.tensor.gdynia.pl>
References: <20090530175239.GA25604@logik.internal.network>
	<20090530144354.2255f722@bhuda.mired.org>
	<20090530191840.GA68514@logik.internal.network>
	<20090530162744.5d77e9d1@bhuda.mired.org>
	<A5BB2D2B836A4438B1B7BD8420FCC6A3@uk.tiscali.intl>
	<h0ehhv$sic$1@ger.gmane.org>
	<b269bc570906061316g37290b5q910da0d3ec266c98@mail.gmail.com>
	<alpine.BSF.2.00.0906070858210.97807@wojtek.tensor.gdynia.pl>
Message-ID: <b269bc570906081211o475d8b16r34fd64388b069a19@mail.gmail.com>

On Sun, Jun 7, 2009 at 12:01 AM, Wojciech Puchar<
wojtek@wojtek.tensor.gdynia.pl> wrote:
>>> (very roughly, in the non-sequential access case) expected to deliver
>>> performance of four drives in a RAID0 array?
>>
>> According to all the Sun documentation, the I/O throughput of a raidz
>> configuration is equal to that of a single drive.
>
> exactly what i say. it's like RAID3. Not RAID5 which have close to n times
> single drive throughput on read and rougly n/4 on writes.
>
>> We remade the pool using 3x 8-drive raidz2 vdevs, and performance has
>> been great (400 MBytes/s write, almost 3 GBytes/s sequential read, 800
>
> why write performance is so slow? in Sun theory it should have the same
> speed as reads. I would say that it should be even better a bit -
filesystem
> get data first in cache and can plan ahead.
>
>> MBytes/s random read).
>
> random read on how big chunks?
>
> Are you sure you get 3GB/s on read? it would mean each drive must be able
to
> do 140MB/s
>
> What disks do you use?

12x 500 GB Seagate EL2 SATA drives, part of their enterprise near-line
storage line.
12x 500 GB WD SATA drives, generic off-the-shelf drives

I re-ran the iozone tests, letting them run to completion, and here are the
results:

The iozone command:  iozone -M -e -+u -T -t <threads> -r 128k -s 40960 -i 0
-i 1 -i 2 -i 8 -+p 70 -C
I ran the command using 32, 64, 128, and 256 for <threads>

Write speeds range from 236 MBytes/sec to 582 MBytes/sec for sequential; and
from 242 MBytes/sec to 550 MBytes/sec for random.

Read speeds range from 3.3 GBytes/sec to 5.5 GBytes/sec for sequential; and
from 1.8 GBytes/sec to 5.5 GBytes/sec for random.

All the gory details are below.

32-threads:  Children see ...  32 initial writers =  582468.13 KB/sec
32-threads:  Parent sees  ...  32 initial writers =  108808.46 KB/sec
64-threads:  Children see ...  64 initial writers =  236144.47 KB/sec
64-threads:  Parent sees  ...  64 initial writers =   86942.94 KB/sec
128-threads: Children see ... 128 initial writers =  284706.68 KB/sec
128-threads: Parent sees  ... 128 initial writers =   10850.40 KB/sec
256-threads: Children see ... 256 initial writers =  258260.59 KB/sec
256-threads: Parent sees  ... 256 initial writers =    9882.16 KB/sec

32-threads:  Children see ...  32 rewriters =  545347.52 KB/sec
32-threads:  Parent sees  ...  32 rewriters =  339308.08 KB/sec
64-threads:  Children see ...  64 rewriters =  419838.51 KB/sec
64-threads:  Parent sees  ...  64 rewriters =  335620.45 KB/sec
128-threads: Children see ... 128 rewriters =  350668.51 KB/sec
128-threads: Parent sees  ... 128 rewriters =  319452.97 KB/sec
256-threads: Children see ... 256 rewriters =  317751.52 KB/sec
256-threads: Parent sees  ... 256 rewriters =  295579.66 KB/sec

32-threads:  Children see ...  32 random writers =  379256.37 KB/sec
32-threads:  Parent sees  ...  32 random writers =   95298.44 KB/sec
64-threads:  Children see ...  64 random writers =  551767.68 KB/sec
64-threads:  Parent sees  ...  64 random writers =  113397.95 KB/sec
128-threads: Children see ... 128 random writers =  241980.60 KB/sec
128-threads: Parent sees  ... 128 random writers =   74584.01 KB/sec
256-threads: Children see ... 256 random writers =  398427.84 KB/sec
256-threads: Parent sees  ... 256 random writers =   20219.56 KB/sec

32-threads:  Children see ...  32 readers = 5023742.86 KB/sec
32-threads:  Parent sees  ...  32 readers = 4661309.72 KB/sec
64-threads:  Children see ...  64 readers = 5516460.71 KB/sec
64-threads:  Parent sees  ...  64 readers = 3949337.61 KB/sec
128-threads: Children see ... 128 readers = 4748635.74 KB/sec
128-threads: Parent sees  ... 128 readers = 3208982.03 KB/sec
256-threads: Children see ... 256 readers = 4358453.38 KB/sec
256-threads: Parent sees  ... 256 readers = 2741593.08 KB/sec

32-threads:  Children see ...  32 re-readers = 5502926.62 KB/sec
32-threads:  Parent sees  ...  32 re-readers = 4650327.75 KB/sec
64-threads:  Children see ...  64 re-readers = 5509400.02 KB/sec
64-threads:  Parent sees  ...  64 re-readers = 4526444.40 KB/sec
128-threads: Children see ... 128 re-readers = 4072363.55 KB/sec
128-threads: Parent sees  ... 128 re-readers = 2840317.47 KB/sec
256-threads: Children see ... 256 re-readers = 3329375.95 KB/sec
256-threads: Parent sees  ... 256 re-readers = 2183894.33 KB/sec

32-threads:  Children see ...  32 random readers = 5555090.45 KB/sec
32-threads:  Parent sees  ...  32 random readers = 4602383.62 KB/sec
64-threads:  Children see ...  64 random readers = 4402270.77 KB/sec
64-threads:  Parent sees  ...  64 random readers = 2059081.52 KB/sec
128-threads: Children see ... 128 random readers = 3070466.93 KB/sec
128-threads: Parent sees  ... 128 random readers =  525076.11 KB/sec
256-threads: Children see ... 256 random readers = 1888676.12 KB/sec
256-threads: Parent sees  ... 256 random readers =  293304.53 KB/sec

32-threads:  Children see ...  32 mixed workload = 3130000.18 KB/sec
32-threads:  Parent sees  ...  32 mixed workload =  123281.78 KB/sec
64-threads:  Children see ...  64 mixed workload = 1587053.33 KB/sec
64-threads:  Parent sees  ...  64 mixed workload =  294586.82 KB/sec
128-threads: Children see ... 128 mixed workload =  807349.95 KB/sec
128-threads: Parent sees  ... 128 mixed workload =   98998.77 KB/sec
256-threads: Children see ... 256 mixed workload =  393469.55 KB/sec
256-threads: Parent sees  ... 256 mixed workload =  112394.90 KB/sec

-- 
Freddie Cash
fjwcash@gmail.com
From pawel.worach at gmail.com  Mon Jun  8 21:11:24 2009
From: pawel.worach at gmail.com (Pawel Worach)
Date: Mon Jun  8 21:11:30 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <20090608.120552.756910862.imp@bsdimp.com>
References: <20090604093831.GE48776@hoeg.nl>
	<20090608.120552.756910862.imp@bsdimp.com>
Message-ID: <d227e09e0906081346j3ee7822n24c1c4968088647d@mail.gmail.com>

On Mon, Jun 8, 2009 at 20:05, M. Warner Losh<imp@bsdimp.com> wrote:
> In message: <20090604093831.GE48776@hoeg.nl>
> ? ? ? ? ? ?Ed Schouten <ed@80386.nl> writes:
> : Good news everyone!
> ...
> : So far we've only done testing on amd64 and i386. A lot of ports are
> : probably still broken. Caveat emptor. Beware of dog. Slippery when wet.
>
> "objects in mirror may be larger than they appear"
>
> Do you have size or run-time performance comparisons yet?
>

Here is a semi-recent build-time benchmark for a stipped down amd64 kernel
clang:
     182.04 real       166.68 user        13.29 sys

gcc:
     217.79 real       211.00 user        13.22 sys

libmicro runtime syscall benchmark results from April:
http://www.vlakno.cz/~pwo/lm/2009-04-11/lm.html

So runtime performance is on par with gcc, code size is a bit bigger
so there is still room for optimization in LLVM.

-- 
Pawel
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  9 10:10:19 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  9 10:10:36 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <d227e09e0906081346j3ee7822n24c1c4968088647d@mail.gmail.com>
References: <20090604093831.GE48776@hoeg.nl>
	<20090608.120552.756910862.imp@bsdimp.com>
	<d227e09e0906081346j3ee7822n24c1c4968088647d@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906091208530.5680@wojtek.tensor.gdynia.pl>

>
> So runtime performance is on par with gcc, code size is a bit bigger

bigger code=lower performance except benchmarks :)
smaller code fits better in cache.

> so there is still room for optimization in LLVM.
>
and for sure there will be some. Good that FreeBSD will have non-GNU 
compiler soon :)
From ed at 80386.nl  Tue Jun  9 10:17:46 2009
From: ed at 80386.nl (Ed Schouten)
Date: Tue Jun  9 10:18:51 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <d227e09e0906081346j3ee7822n24c1c4968088647d@mail.gmail.com>
References: <20090604093831.GE48776@hoeg.nl>
	<20090608.120552.756910862.imp@bsdimp.com>
	<d227e09e0906081346j3ee7822n24c1c4968088647d@mail.gmail.com>
Message-ID: <20090609101745.GE48776@hoeg.nl>

* Pawel Worach <pawel.worach@gmail.com> wrote:
> So runtime performance is on par with gcc, code size is a bit bigger
> so there is still room for optimization in LLVM.

I don't agree on the code size. Code size is comparable. I just did a
quick ls through /bin. There also seem to be a lot of cases where Clang
generates smaller binaries.

Some time ago the binaries were indeed a lot bigger, but that turned out
to be a bug in Clang, where a compiler flag had a wrong default value,
namely the flag that determined whether zero-initialized data had to go
in BSS or not.

-- 
 Ed Schouten <ed@80386.nl>
 WWW: http://80386.nl/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090609/c847a7b6/attachment.pgp
From dan.naumov at gmail.com  Tue Jun  9 10:10:17 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Tue Jun  9 11:35:18 2009
Subject: sysinstall, GJOURNAL and ZFS
Message-ID: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>

Hello list

Let me preface this by saying that I do not have coding
knowledge/experience, but I am willing to donate my time to help test
things if somebody is already working on this. Hopefully, this will
prevent most of the potential "feel free to submit patches" responses
:)

Is there any work going on to make sysinstall recognize and abe able
to create and work with GJOURNAL and ZFS? In the days of 1,5-2,0
terabyte harddrives, UFS2 + SoftUpdates simply doesn't cut it anymore,
but since this is the only thing you can do from within sysinstall, it
is what the vast majority of people giving FreeBSD a try are going to
end up with. If you tell a Windows or a Linux user that having a
sudden power loss or having to pull the plug on a locked system can
result in an unbootable FreeBSD system, a very large portion of them
is going to run away screaming, never giving FreeBSD a second look.
And yes, I know that the same things can theoretically render Windows
or Linux unbootable, but let's be honest here, it is a LOT more likely
to happen to an UFS2 + SoftUpdates system than NTFS or EXT3/EXT4.

This calls for more robust default options/settings to be presented by
the sysinstall to the new user. GJOURNAL and ZFS present themselves as
the most sensible options in this case.


Sincerely,
- Dan Naumov
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  9 14:36:32 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  9 14:36:45 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>

> Is there any work going on to make sysinstall recognize and abe able
> to create and work with GJOURNAL and ZFS? In the days of 1,5-2,0
> terabyte harddrives, UFS2 + SoftUpdates simply doesn't cut it anymore,
UFS2+SoftUpdates works fine on properly configured UFS2 - and very fast.

Why you need sysinstall AT ALL?


use any CD/DVD distro like FreeSBIE, or even better - the one that 
loads itself to RAM and no longer needs HDD, or even FreeBSD already 
installed on Pendrive.

Boot it, then partition/newfs your disk(s) whatever you like, mount 
target, then use 
install.sh scripts from FreeBSD install CD to install system

this way

DESTDIR=/target ./install.sh

in every directory

then install bootblock and edit fstab, your system will boot.


Sysinstall is to help beginners, but do beginners need so complex setups?
From dan.naumov at gmail.com  Tue Jun  9 14:57:32 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Tue Jun  9 15:23:05 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
Message-ID: <cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>

> UFS2+SoftUpdates works fine on properly configured UFS2 - and very fast.
Yes, UFS2+SoftUpdates is very fast, however, in the case of a power
loss or having to pull the plug on a locked up system, it has a
noticeably higher chance of leaving you with an unbootable system than
if you were using Linux with ext3/ext4 or Windows with NTFS.


> Why you need sysinstall AT ALL?
I am well capable of doing a textmode installation using console
commands only, so _I_ don't. However, sysinstall is what the majority
of new users are presented with when giving FreeBSD a try. A new user
who loses data (let alone has his system become unbootable) on an
unclean shutdown is a very unhappy user. A very unhappy user is a user
likely to go back to what he was using before trying FreeBSD. Happy
users means more users, more users means more developers, more
developers means more active development, which is surely not a bad
thing. DO NOT underestimate the value of a good installer with good
defaults.


<SNIP>
>
> then install bootblock and edit fstab, your system will boot.
Having to do manual operating system installations in the year 2009 is
a broken installation process.


> Sysinstall is to help beginners, but do beginners need so complex setups?
A user who wants his data to be well protected and his system to have
high resistance to becoming unbootable in the case of a power loss is
a user with completely reasonable expectations, not a user with a
"complex setup".


- Dan Naumov
From vince at unsane.co.uk  Tue Jun  9 15:50:55 2009
From: vince at unsane.co.uk (Vincent Hoffman)
Date: Tue Jun  9 15:51:07 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
Message-ID: <4A2E84DC.1010900@unsane.co.uk>

On 9/6/09 15:57, Dan Naumov wrote:
>> UFS2+SoftUpdates works fine on properly configured UFS2 - and very fast.
>>     
> Yes, UFS2+SoftUpdates is very fast, however, in the case of a power
> loss or having to pull the plug on a locked up system, it has a
> noticeably higher chance of leaving you with an unbootable system than
> if you were using Linux with ext3/ext4 or Windows with NTFS.
>
>
>   
Interestingly in my experience its been the opposite, I've lost a few
ext3 filesystems though bad power, same for NTFS (NT4, less so with
200x) but as yet never for ufs2 (fsck has always fixed it.)

That said, there have been a few projects to update/replace/whatever
sysinstall, look at the desktopBSD installer (bsdinstaller) and
finstall. I'm not sure what the status of either of these 2 are though.

Vince

From snb at freebsd.org  Tue Jun  9 17:21:45 2009
From: snb at freebsd.org (Nick Barkas)
Date: Tue Jun  9 17:21:51 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
Message-ID: <20090609172142.GA92146@ebi.local>

On Tue, Jun 09, 2009 at 04:57:30PM +0200, Dan Naumov wrote:
> > UFS2+SoftUpdates works fine on properly configured UFS2 - and very fast.
> Yes, UFS2+SoftUpdates is very fast, however, in the case of a power
> loss or having to pull the plug on a locked up system, it has a
> noticeably higher chance of leaving you with an unbootable system than
> if you were using Linux with ext3/ext4 or Windows with NTFS.

Can you back this up? I cannot recall having ever rendered a FreeBSD
system unbootable due to UFS/UFS2 problems after a power failure or
crash. I once had a problem with snapshots that made background fsck
fail and crash the system, but it was fixable by booting single user and
running fsck manually. This was a couple of years ago, and I think the
problem has since been fixed.

ZFS at least probably is not able to replace UFS2 for everyone, at this
time, anyway. Perhaps gjournal can be a replacement for softupdates for
many people who do still need UFS2, but I'm not sure. I think in any
case, any existing bugs that cause UFS2+softupdates to catastrophically
fail in the event of power failures or system crashes need to be fixed.
Making it easier for users to install a system with ZFS or gjournal does
nothing for those who have existing systems with UFS2+softupdates, and
those who cannot use ZFS or gjournal in the near future for whatever
reason.

That being said, I do agree that being able to support ZFS and gjournal
in sysinstall or an alternative installer would be great. 

Nick
From sullrich at gmail.com  Tue Jun  9 17:26:45 2009
From: sullrich at gmail.com (Scott Ullrich)
Date: Tue Jun  9 17:26:52 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <4A2E84DC.1010900@unsane.co.uk>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com> 
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl> 
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com> 
	<4A2E84DC.1010900@unsane.co.uk>
Message-ID: <d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>

On Tue, Jun 9, 2009 at 11:50 AM, Vincent Hoffman<vince@unsane.co.uk> wrote:
[snip]
>
> That said, there have been a few projects to update/replace/whatever
> sysinstall, look at the desktopBSD installer (bsdinstaller) and
> finstall. I'm not sure what the status of either of these 2 are though.

I was holding off on announcing this publicly but I guess now is a good time.

Currently we are building FreeBSD-CURRENT + BSDInstaller snapshots
every 12 hours or so at:
http://snapshots.pfsense.org/FreeBSD_8_0/?C=M;O=D

More information about BSDInstaller / FreeBSD integration is located
here: http://wiki.freebsd.org/FreeBSD/BSDInstaller2009

If folks would like to help out with the remaining needed items that
would be awesome.

Scott
From sullrich at gmail.com  Tue Jun  9 17:31:50 2009
From: sullrich at gmail.com (Scott Ullrich)
Date: Tue Jun  9 17:32:34 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906091017k2482bfga6430d8f52a22a2c@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com> 
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl> 
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com> 
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com> 
	<cf9b1ee00906091017k2482bfga6430d8f52a22a2c@mail.gmail.com>
Message-ID: <d5992baf0906091031u23403621sd65eb24752b9ecdd@mail.gmail.com>

On Tue, Jun 9, 2009 at 1:17 PM, Dan Naumov<dan.naumov@gmail.com> wrote:
> Great! I am downloading
> http://snapshots.pfsense.org/FreeBSD_8_0/FreeBSD-20090608-1522-8.0-CURRENT.iso.gz
> as we speak and will give it a whirl within the next few days. Any
> plans to do similar snapshot builds of -STABLE?

I had not planned on it but I guess that some bending of my arm might
be in order.

I'll look into it.   We just retired a builder box for pfSense that I
might be able to use.

Scott
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  9 17:38:03 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  9 17:38:13 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <4A2E84DC.1010900@unsane.co.uk>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<4A2E84DC.1010900@unsane.co.uk>
Message-ID: <alpine.BSF.2.00.0906091932390.7529@wojtek.tensor.gdynia.pl>

>>
>>
> Interestingly in my experience its been the opposite, I've lost a few
> ext3 filesystems though bad power, same for NTFS (NT4, less so with
> 200x) but as yet never for ufs2 (fsck has always fixed it.)

In worse cases it required manual attention :) UFS is used and improved 
over 20 years, it's SIMPLE and practically bug-free now, except snapshots.

I think it could be improved a bit more to support multiterabyte disks 
better. Like more blocks allowed per cylgroup and few other changes that 
will result in no more than 1000 cylgroups on that devices. This will 
improve fsck times a lot.

Still it's well usable on 2TB disks.
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  9 17:44:24 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  9 17:44:37 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <20090609172142.GA92146@ebi.local>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
Message-ID: <alpine.BSF.2.00.0906091938330.7529@wojtek.tensor.gdynia.pl>

>> noticeably higher chance of leaving you with an unbootable system than
>> if you were using Linux with ext3/ext4 or Windows with NTFS.
>
> Can you back this up? I cannot recall having ever rendered a FreeBSD
> system unbootable due to UFS/UFS2 problems after a power failure or

I can confirm the opposite. I did really bad things with disk drives, 
including accidentally disconnecting SATA cable on working system that was 
doing lots of filesystem writes. NOTHING wrong happened.

I always use softupdates, including / partition (which usually is my only 
partition anyway :)

Once i have failed hardware which sometimes was writting rubbish to disk 
instead of proper data. after discovering this there was a bit mess but 
ONLY in files and directories i was writing to!!! fsck with manual 
attention repaired it all without problems. After fixing hardware problems 
and few files i used it without change.

I can't count how many times i ended with completely unreadable ext2, ext3 
and reiserfs filesystem, the last being really kind of destruction ;)
From sullrich at gmail.com  Tue Jun  9 17:48:58 2009
From: sullrich at gmail.com (Scott Ullrich)
Date: Tue Jun  9 17:49:05 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906091046p409ca227p12699a762716b9b2@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com> 
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl> 
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com> 
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com> 
	<cf9b1ee00906091017k2482bfga6430d8f52a22a2c@mail.gmail.com> 
	<d5992baf0906091031u23403621sd65eb24752b9ecdd@mail.gmail.com> 
	<cf9b1ee00906091046p409ca227p12699a762716b9b2@mail.gmail.com>
Message-ID: <d5992baf0906091048k59b7d390q913e834f82e9dba2@mail.gmail.com>

On Tue, Jun 9, 2009 at 1:46 PM, Dan Naumov<dan.naumov@gmail.com> wrote:
> What arch are these snapshots, are they amd64 or i386? Speaking of
> -STABLE snapshots, since they are a more slowly moving target than
> -CURRENT, 1 snapshot every week or so would definately be enough :)

These are i386 FreeBSD-8-CURRENT.  We build every 12 hours or so for
BSDInstaller testing.   Instead of me having to roll an ISO after
every change I just wait a bit and test.  Works well for me and
CPU/Disk space is cheap.

Scott
From sullrich at gmail.com  Tue Jun  9 17:50:34 2009
From: sullrich at gmail.com (Scott Ullrich)
Date: Tue Jun  9 17:50:41 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906091046p409ca227p12699a762716b9b2@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com> 
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl> 
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com> 
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com> 
	<cf9b1ee00906091017k2482bfga6430d8f52a22a2c@mail.gmail.com> 
	<d5992baf0906091031u23403621sd65eb24752b9ecdd@mail.gmail.com> 
	<cf9b1ee00906091046p409ca227p12699a762716b9b2@mail.gmail.com>
Message-ID: <d5992baf0906091048k59b7d390q913e834f82e9dba2@mail.gmail.com>

On Tue, Jun 9, 2009 at 1:46 PM, Dan Naumov<dan.naumov@gmail.com> wrote:
> What arch are these snapshots, are they amd64 or i386? Speaking of
> -STABLE snapshots, since they are a more slowly moving target than
> -CURRENT, 1 snapshot every week or so would definately be enough :)

These are i386 FreeBSD-8-CURRENT.  We build every 12 hours or so for
BSDInstaller testing.   Instead of me having to roll an ISO after
every change I just wait a bit and test.  Works well for me and
CPU/Disk space is cheap.

Scott
From dan.naumov at gmail.com  Tue Jun  9 17:17:20 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Tue Jun  9 18:03:25 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>
Message-ID: <cf9b1ee00906091017k2482bfga6430d8f52a22a2c@mail.gmail.com>

Great! I am downloading
http://snapshots.pfsense.org/FreeBSD_8_0/FreeBSD-20090608-1522-8.0-CURRENT.iso.gz
as we speak and will give it a whirl within the next few days. Any
plans to do similar snapshot builds of -STABLE?

- Dan Naumov


On Tue, Jun 9, 2009 at 7:58 PM, Scott Ullrich<sullrich@gmail.com> wrote:
> On Tue, Jun 9, 2009 at 11:50 AM, Vincent Hoffman<vince@unsane.co.uk> wrote:
> [snip]
>>
>> That said, there have been a few projects to update/replace/whatever
>> sysinstall, look at the desktopBSD installer (bsdinstaller) and
>> finstall. I'm not sure what the status of either of these 2 are though.
>
> I was holding off on announcing this publicly but I guess now is a good time.
>
> Currently we are building FreeBSD-CURRENT + BSDInstaller snapshots
> every 12 hours or so at:
> http://snapshots.pfsense.org/FreeBSD_8_0/?C=M;O=D
>
> More information about BSDInstaller / FreeBSD integration is located
> here: http://wiki.freebsd.org/FreeBSD/BSDInstaller2009
>
> If folks would like to help out with the remaining needed items that
> would be awesome.
>
> Scott
>
From dan.naumov at gmail.com  Tue Jun  9 17:36:35 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Tue Jun  9 18:03:39 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <20090609172142.GA92146@ebi.local>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
Message-ID: <cf9b1ee00906091036q2ef84980gd8a046e21107782c@mail.gmail.com>

I know some might dismiss my personal experience as "anecdotal", but
yes I have had that happen to me twice. Some googling will show that I
am not the only one to ever experience fatal UFS+softupdates failures.
I even ran into a page from 2006 (I think), where a fellow wrote some
code which would write data to an UFS2 partition that would
consistently cause fsck to fail. This actually brings up to another
related point, having to actually deal with fsck. On
filesystems/volumes of today, which can easily span 10tb+ in a
production environment, having to deal with fsck times is a complete
no-go.

One could of course argue that environments where 10tb+ volumes are
used right now are special cases, where the administrator would know
of these issues and take appropriate measures to deal with it (UPS,
custom installation with gjournal, ZFS, etc), my point still stands.
Right now, anyone can walk into a computer store and purchase a
consumer 1,5tb or 2tb drive. Have you tried running fsck on a 2tb
filesystem? It's not fun. And this will only get worse with time as
drive sizes continue to grow.

- Dan Naumov





On Tue, Jun 9, 2009 at 8:21 PM, Nick Barkas<snb@freebsd.org> wrote:
> Can you back this up? I cannot recall having ever rendered a FreeBSD
> system unbootable due to UFS/UFS2 problems after a power failure or
> crash. I once had a problem with snapshots that made background fsck
> fail and crash the system, but it was fixable by booting single user and
> running fsck manually. This was a couple of years ago, and I think the
> problem has since been fixed.
>
> ZFS at least probably is not able to replace UFS2 for everyone, at this
> time, anyway. Perhaps gjournal can be a replacement for softupdates for
> many people who do still need UFS2, but I'm not sure. I think in any
> case, any existing bugs that cause UFS2+softupdates to catastrophically
> fail in the event of power failures or system crashes need to be fixed.
> Making it easier for users to install a system with ZFS or gjournal does
> nothing for those who have existing systems with UFS2+softupdates, and
> those who cannot use ZFS or gjournal in the near future for whatever
> reason.
>
> That being said, I do agree that being able to support ZFS and gjournal
> in sysinstall or an alternative installer would be great.
>
> Nick
>
From manolis at FreeBSD.org  Tue Jun  9 18:11:30 2009
From: manolis at FreeBSD.org (Manolis Kiagias)
Date: Tue Jun  9 18:11:38 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
Message-ID: <4A2EA355.6010800@FreeBSD.org>

Dan Naumov wrote:
> Hello list
>
> Let me preface this by saying that I do not have coding
> knowledge/experience, but I am willing to donate my time to help test
> things if somebody is already working on this. Hopefully, this will
> prevent most of the potential "feel free to submit patches" responses
> :)
>
> Is there any work going on to make sysinstall recognize and abe able
> to create and work with GJOURNAL and ZFS? 

You can still install a gjournal based system fairly easily, just follow
this:

http://www.freebsd.org/doc/en/articles/gjournal-desktop/index.html
From dan.naumov at gmail.com  Tue Jun  9 17:46:53 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Tue Jun  9 18:23:15 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <d5992baf0906091031u23403621sd65eb24752b9ecdd@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>
	<cf9b1ee00906091017k2482bfga6430d8f52a22a2c@mail.gmail.com>
	<d5992baf0906091031u23403621sd65eb24752b9ecdd@mail.gmail.com>
Message-ID: <cf9b1ee00906091046p409ca227p12699a762716b9b2@mail.gmail.com>

What arch are these snapshots, are they amd64 or i386? Speaking of
-STABLE snapshots, since they are a more slowly moving target than
-CURRENT, 1 snapshot every week or so would definately be enough :)

- Dan Naumov



On Tue, Jun 9, 2009 at 8:31 PM, Scott Ullrich<sullrich@gmail.com> wrote:
> On Tue, Jun 9, 2009 at 1:17 PM, Dan Naumov<dan.naumov@gmail.com> wrote:
>> Great! I am downloading
>> http://snapshots.pfsense.org/FreeBSD_8_0/FreeBSD-20090608-1522-8.0-CURRENT.iso.gz
>> as we speak and will give it a whirl within the next few days. Any
>> plans to do similar snapshot builds of -STABLE?
>
> I had not planned on it but I guess that some bending of my arm might
> be in order.
>
> I'll look into it. ? We just retired a builder box for pfSense that I
> might be able to use.
>
> Scott
From sthaug at nethelp.no  Tue Jun  9 18:24:34 2009
From: sthaug at nethelp.no (sthaug@nethelp.no)
Date: Tue Jun  9 18:58:46 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <20090609172142.GA92146@ebi.local>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
Message-ID: <20090609.195750.41709103.sthaug@nethelp.no>

> Can you back this up? I cannot recall having ever rendered a FreeBSD
> system unbootable due to UFS/UFS2 problems after a power failure or
> crash. I once had a problem with snapshots that made background fsck
> fail and crash the system, but it was fixable by booting single user and
> running fsck manually. This was a couple of years ago, and I think the
> problem has since been fixed.

I've had several cases that needed manual fsck. After I turned off
background fsck, the problems stopped. These days background_fsck="NO"
is a standard part of my rc.conf.

Steinar Haug, Nethelp consulting, sthaug@nethelp.no
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  9 20:07:42 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  9 20:07:48 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <20090609.195750.41709103.sthaug@nethelp.no>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
Message-ID: <alpine.BSF.2.00.0906092205350.8273@wojtek.tensor.gdynia.pl>

>> problem has since been fixed.
>
> I've had several cases that needed manual fsck. After I turned off
> background fsck, the problems stopped. These days background_fsck="NO"
> is a standard part of my rc.conf.
and mine.

actually snapshots doesn't work on large partitions - could simply crash. 
that's why i wrote that UFS has practically no bugs, except snapshots.

But i don't see it as a problem. i don't like my system to start unchecked 
and then be "background" checked. this background checking slows down 
system so much and takes more time than foreground checking.

FreeBSD really doesn't crash every day, so what a problem to just do fsck 
at start and wait a bit.
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  9 20:10:32 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  9 20:10:39 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906091036q2ef84980gd8a046e21107782c@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<cf9b1ee00906091036q2ef84980gd8a046e21107782c@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906092208100.8273@wojtek.tensor.gdynia.pl>

> filesystems/volumes of today, which can easily span 10tb+ in a
> production environment, having to deal with fsck times is a complete
> no-go.
>
just use large block sizes are really small amount of inodes. it's 
unlikely that you will fill such huge FS with mostly small files, so 
larger blocks are not wasting space.

Just DON'T USE defaults.

fsck takes <5 minutes to check 1TB drive with -b 32768 and -i 524288

> One could of course argue that environments where 10tb+ volumes are
> used right now are special cases, where the administrator would know

No it's not special case. Just needs to be handled properly.

From randy at psg.com  Tue Jun  9 20:35:07 2009
From: randy at psg.com (Randy Bush)
Date: Tue Jun  9 20:47:12 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <20090609.195750.41709103.sthaug@nethelp.no>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
Message-ID: <m2zlchdu2x.wl%randy@psg.com>

> I've had several cases that needed manual fsck. After I turned off
> background fsck, the problems stopped. These days background_fsck="NO"
> is a standard part of my rc.conf.

don't be so negative, steinar, balance it with

fsck_y_enable=YES

randy
From kraduk at googlemail.com  Tue Jun  9 20:58:28 2009
From: kraduk at googlemail.com (krad)
Date: Tue Jun  9 21:08:44 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <alpine.BSF.2.00.0906092208100.8273@wojtek.tensor.gdynia.pl>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com><alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl><cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com><20090609172142.GA92146@ebi.local><cf9b1ee00906091036q2ef84980gd8a046e21107782c@mail.gmail.com>
	<alpine.BSF.2.00.0906092208100.8273@wojtek.tensor.gdynia.pl>
Message-ID: <EB3A708B8AF44BA3A1C3639F43CA4D0E@uk.tiscali.intl>

Hmm I disagree about large fs have large files. We have inherited quite a
few mail servers at work with 1 TB + fs. They had 10 of millions of files.
When we had a failure and had one reboot it was a nightmare, took ages to
fix. Needless to say this is all on a zfs backed nfs filer now thank god

I do take your point though in many cases what you say would be a good idea.


-----Original Message-----
From: owner-freebsd-hackers@freebsd.org
[mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of Wojciech Puchar
Sent: 09 June 2009 21:10
To: Dan Naumov
Cc: Nick Barkas; freebsd-current@freebsd.org; freebsd-hackers@freebsd.org
Subject: Re: sysinstall, GJOURNAL and ZFS

> filesystems/volumes of today, which can easily span 10tb+ in a
> production environment, having to deal with fsck times is a complete
> no-go.
>
just use large block sizes are really small amount of inodes. it's 
unlikely that you will fill such huge FS with mostly small files, so 
larger blocks are not wasting space.

Just DON'T USE defaults.

fsck takes <5 minutes to check 1TB drive with -b 32768 and -i 524288

> One could of course argue that environments where 10tb+ volumes are
> used right now are special cases, where the administrator would know

No it's not special case. Just needs to be handled properly.

_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

From wojtek at wojtek.tensor.gdynia.pl  Tue Jun  9 21:23:03 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun  9 21:23:15 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <EB3A708B8AF44BA3A1C3639F43CA4D0E@uk.tiscali.intl>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com><alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl><cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com><20090609172142.GA92146@ebi.local><cf9b1ee00906091036q2ef84980gd8a046e21107782c@mail.gmail.com>
	<alpine.BSF.2.00.0906092208100.8273@wojtek.tensor.gdynia.pl>
	<EB3A708B8AF44BA3A1C3639F43CA4D0E@uk.tiscali.intl>
Message-ID: <alpine.BSF.2.00.0906092322240.8915@wojtek.tensor.gdynia.pl>

> Hmm I disagree about large fs have large files. We have inherited quite a
> few mail servers at work with 1 TB + fs. They had 10 of millions of files.

still 100kB/file. It's fine to use -b 32768 -f 4096 -i 65536 (or 32768 at 
least)

From rdivacky at FreeBSD.org  Wed Jun 10 08:52:57 2009
From: rdivacky at FreeBSD.org (Roman Divacky)
Date: Wed Jun 10 08:53:03 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <DF7D22CF-94C1-4B27-A216-E4AD5FEBCA53@cederstrand.dk>
References: <20090604093831.GE48776@hoeg.nl>
	<31BD4D08-6558-46FF-9B93-CF8249AAC461@cederstrand.dk>
	<20090604123801.GA34971@freebsd.org>
	<DF7D22CF-94C1-4B27-A216-E4AD5FEBCA53@cederstrand.dk>
Message-ID: <20090610085138.GA7459@freebsd.org>

On Fri, Jun 05, 2009 at 09:45:44AM +0200, Erik Cederstrand wrote:
> Hi Roman
> 
> Den 04/06/2009 kl. 14.38 skrev Roman Divacky:
> >
> >you could use llvm-ld (see the wiki for instructions how to do it).  
> >there's also some
> >effort to make gnu ld usable with llvm LTO and I guess the patch  
> >could be backported
> >to our ld. I guess
> 
> As I understand the reply from Eli Friedman[1] this is not possible  
> without at least some hacking.
> 
> The LTO work in GNU ld[2] is under GPLv3[3], as is gold[4], which  
> makes backporting patches a sticky issue.

we can ask the person who did the gnu ld work to relicense the one patch
under gplv2
From des at des.no  Wed Jun 10 10:05:03 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Wed Jun 10 10:05:11 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <20090609.195750.41709103.sthaug@nethelp.no> (sthaug@nethelp.no's
	message of "Tue, 09 Jun 2009 19:57:50 +0200 (CEST)")
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
Message-ID: <86hbyowgj6.fsf@ds4.des.no>

sthaug@nethelp.no writes:
> I've had several cases that needed manual fsck. After I turned off
> background fsck, the problems stopped. These days background_fsck="NO"
> is a standard part of my rc.conf.

Hear, hear.

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From marck at rinet.ru  Wed Jun 10 10:31:33 2009
From: marck at rinet.ru (Dmitry Morozovsky)
Date: Wed Jun 10 10:32:27 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <86hbyowgj6.fsf@ds4.des.no>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
	<86hbyowgj6.fsf@ds4.des.no>
Message-ID: <alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>

On Wed, 10 Jun 2009, Dag-Erling Sm?rgrav wrote:

DS> sthaug@nethelp.no writes:
DS> > I've had several cases that needed manual fsck. After I turned off
DS> > background fsck, the problems stopped. These days background_fsck="NO"
DS> > is a standard part of my rc.conf.
DS> 
DS> Hear, hear.

Well, I can see at least one rather big problem with bgfsck (or with snapshots 
to be more precise): inappropriate time of file system lock on snapshot 
creation. On not-too-big 300G ufs2 not-too-heavy loaded snapshot creation time 
is 20+ minutes, and 5+ from that file system blocked even on reads.  This looks 
unacceptable for me for any real use.

-- 
Sincerely,
D.Marck                                     [DM5020, MCK-RIPE, DM3-RIPN]
[ FreeBSD committer:                                 marck@FreeBSD.org ]
------------------------------------------------------------------------
*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru ***
------------------------------------------------------------------------
From James.McPherson at Sun.COM  Wed Jun 10 04:19:18 2009
From: James.McPherson at Sun.COM (James C. McPherson)
Date: Wed Jun 10 11:39:53 2009
Subject: Only _3_ days left for earlybird registration to Kernel Conference
 Australia
Message-ID: <20090610141849.000030f5@blinder>



Hi everybody,
just a short reminder that if you want to come to Kernel
Conference Australia at the earlybird price of $195, then
you've got until the end of this Friday, 12 June, to get
your registration happening.

If you're interested in any of these areas:

* linux kernel crypto services
* ZFS
* cross-architecture OS and driver porting
* packet filtering and QOS
* TCP/IP protocol security
* bug finding tools for OpenSource Operating Systems
* network virtualisation, or
* how a DB engine can really hurt your system

then Kernel Conference Australia is for you.

Dates:    15 - 17 July
Venue:    Queensland Brain Institute, UQ


The student price is still $95, too.

For full abstracts please see http://wikis.sun.com/display/KCA2009/KCA2009+Conference+Agenda

For the conference homepage, see
http://au.sun.com/sunnews/events/2009/kernel/


And for registration,   Go Without Delay To
https://www.conveneit.com/secure/sun/kernel_jul_09/


I look forward to seeing you there.


Cheers, 
James C. McPherson
--
Senior Kernel Software Engineer, Solaris
Sun Microsystems
http://blogs.sun.com/jmcp	http://www.jmcp.homeunix.com/blog
Kernel Conference Australia - http://au.sun.com/sunnews/events/2009/kernel
From wojtek at wojtek.tensor.gdynia.pl  Wed Jun 10 11:47:06 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Wed Jun 10 11:47:19 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
	<86hbyowgj6.fsf@ds4.des.no>
	<alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>
Message-ID: <alpine.BSF.2.00.0906101346040.16323@wojtek.tensor.gdynia.pl>

> to be more precise): inappropriate time of file system lock on snapshot
> creation. On not-too-big 300G ufs2 not-too-heavy loaded snapshot creation time
> is 20+ minutes, and 5+ from that file system blocked even on reads.  This looks
> unacceptable for me for any real use.

that's why i disable it. If you sum up time of total blocked and time of 
almost-blocked you will end with much more than fsck normally use with 
foreground check.


From dan.naumov at gmail.com  Wed Jun 10 12:04:54 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Wed Jun 10 12:25:38 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <alpine.BSF.2.00.0906101346040.16323@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
	<86hbyowgj6.fsf@ds4.des.no>
	<alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>
	<alpine.BSF.2.00.0906101346040.16323@wojtek.tensor.gdynia.pl>
Message-ID: <cf9b1ee00906100504p31de4d4dj1ddd4c54b95f95de@mail.gmail.com>

And things like these are amongst the reasons why I want to see newer
options be presented and offered to the user during installation
process. Default installation options (and the ONLY options presented
by sysinstall) that result in enormous snapshot creation times and
long fsck times are just not good enough in the year 2009. I fully
understand the reasons behind using something "truly tested and proven
by time" as the default, but presenting the newer options to the user
during installation process as well gives these newer options
(UFS2+GJournal and ZFS in this case) a better exposure, resulting in
more testing, resulting in these new features getting their quirks
ironed out faster and resulting in these new features getting the
"truly tested and proven by time" stamp of approval faster. I think we
can all agree this would be a good thing?

Sincerely
- Dan Naumov



On Wed, Jun 10, 2009 at 2:46 PM, Wojciech
Puchar<wojtek@wojtek.tensor.gdynia.pl> wrote:
>> to be more precise): inappropriate time of file system lock on snapshot
>> creation. On not-too-big 300G ufs2 not-too-heavy loaded snapshot creation
>> time
>> is 20+ minutes, and 5+ from that file system blocked even on reads. ?This
>> looks
>> unacceptable for me for any real use.
>
> that's why i disable it. If you sum up time of total blocked and time of
> almost-blocked you will end with much more than fsck normally use with
> foreground check.
>
>
>
From wojtek at wojtek.tensor.gdynia.pl  Wed Jun 10 13:11:47 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Wed Jun 10 13:12:01 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906100504p31de4d4dj1ddd4c54b95f95de@mail.gmail.com>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
	<86hbyowgj6.fsf@ds4.des.no>
	<alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>
	<alpine.BSF.2.00.0906101346040.16323@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906100504p31de4d4dj1ddd4c54b95f95de@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906101511110.16752@wojtek.tensor.gdynia.pl>

> during installation process as well gives these newer options
> (UFS2+GJournal and ZFS in this case) a better exposure, resulting in
> more testing, resulting in these new features getting their quirks
> ironed out faster and resulting in these new features getting the
> "truly tested and proven by time" stamp of approval faster. I think we
> can all agree this would be a good thing?
so fix sysinstall and do sent-pr.

From dan.naumov at gmail.com  Wed Jun 10 13:15:57 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Wed Jun 10 13:43:02 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <alpine.BSF.2.00.0906101511110.16752@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
	<86hbyowgj6.fsf@ds4.des.no>
	<alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>
	<alpine.BSF.2.00.0906101346040.16323@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906100504p31de4d4dj1ddd4c54b95f95de@mail.gmail.com>
	<alpine.BSF.2.00.0906101511110.16752@wojtek.tensor.gdynia.pl>
Message-ID: <cf9b1ee00906100615j70992238j5de9167d7cafce55@mail.gmail.com>

On Tue, Jun 9, 2009 at 12:42 PM, Dan Naumov<dan.naumov@gmail.com> wrote:
> Hello list
>
> Let me preface this by saying that I do not have coding
> knowledge/experience, but I am willing to donate my time to help test
> things if somebody is already working on this. Hopefully, this will
> prevent most of the potential "feel free to submit patches" responses
> :)
>


>
> so fix sysinstall and do sent-pr.
>
>
From tim at tangobravo.co.uk  Wed Jun 10 14:01:03 2009
From: tim at tangobravo.co.uk (Tim Borgeaud)
Date: Wed Jun 10 15:55:48 2009
Subject: Device drivers, mmap/munmap and freeing memory
Message-ID: <4A2FB742.6080307@tangobravo.co.uk>

I'm currently working on some wrappers and compatibility functions for 
allowing USB Linux device driver code to be compiled to create a FreeBSD 
driver module. I.e. extending the work of Luigi Rizzo (linux-kmod-compat) 
and/or the compatibility code now present in the FreeBSD USB stack.

This approach, for creating drivers from Linux source, will probably be 
moved into userland in one way or another, but for now I'm going to see if 
I can make some progress by getting some Linux webcam drivers working.

The problem I'm currently wrestling with is how to manage the emulation of 
Linux's mmapping.

Linux driver code expects to be able to use some callback functions that 
are invoked when certain mmap related events occur. Drivers appear to use 
an open/close pair of functions that are called when a processes starts or 
stops using mapped memory. These are typically used to maintain a 
reference count for the memory.

As far as I can tell, in FreeBSD, a vm_object_t also contains a similar 
reference count. The vm subsystem cleans up vm objects (and vm_map entries 
etc) when the reference count falls to zero.

The trouble I have is that it appears that I'm going to need some way to 
let Linux driver code know whether or not some memory is still in use, 
i.e. whether the FreeBSD system still holds a vm_object for the mmapped 
memory.

Ideally, I could invoke the Linux driver functions appropriately (when the 
vm subsystem increments or decrements reference counts for a vm_object). 
However, it should be enough just to make sure that the Linux driver close 
callback function is invoked when mapped memory has been unmapped.

As far as I can tell, closing mapped file descriptors should not effect 
the reference count. It appears that it is quite legitimate to access 
mapped memory after the corresponing file descriptor has been closed.

I'm wondering if there is any way to figure out whether some memory is 
still mapped (by the vm subsystem), whether or not a driver could be 
informed about munmapping (or forking etc) or what the possible effects 
would be if memory that has been mmapped is freed (while it may be in use).

I suspect that applications will simply open, mmap and then close (and 
exit). So I think it would not be unreasonable to cause applications that 
work in some other way to fail. However, I also suspect that the failure 
resulting from attempts to access memory that has just been freed by a 
driver may be worse than a crashed application.

Tim
From cdillon at wolves.k12.mo.us  Wed Jun 10 16:11:26 2009
From: cdillon at wolves.k12.mo.us (Chris Dillon)
Date: Wed Jun 10 16:11:32 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>
References: <alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<20090609172142.GA92146@ebi.local>
	<20090609.195750.41709103.sthaug@nethelp.no>
	<86hbyowgj6.fsf@ds4.des.no>
	<alpine.BSF.2.00.0906101428591.49870@woozle.rinet.ru>
Message-ID: <20090610105515.138862z6opgohgtf@www.wolves.k12.mo.us>

Quoting Dmitry Morozovsky <marck@rinet.ru>:

> Well, I can see at least one rather big problem with bgfsck (or with  
> snapshots to be more precise): inappropriate time of file system  
> lock on snapshot creation. On not-too-big 300G ufs2 not-too-heavy  
> loaded snapshot creation time is 20+ minutes, and 5+ from that file  
> system blocked even on reads.  This looks unacceptable for me for  
> any real use.

The snapshot time depends heavily on the I/O throughput of your disk  
subsystem.  On a several year old system with 5 x 72GB 15KRPM U320  
SCSI drives in a RAID5 array, a fairly well loaded 260GB filesystem  
(90GB used, 354K out of 8M inodes used, and several hundred MB to a GB  
of changes per day) completes a snapshot in exactly 2 minutes.  2  
minutes is still too long to be blocking I/O in the middle of the day  
when it is being actively used, so I just take 1 snapshot per day  
while it is idle.  I would love to put ZFS on this system so that I  
could have finer grained snapshots, but I need user quota support  
which our ZFS currently lacks.

-- 

Chris Dillon - NetEng/SysAdm
Reeds Spring R-IV School District
Technology Department
175 Elementary Rd.
Reeds Spring, MO  65737
Voice: 417-272-8266   Fax: 417-272-0015


From yuri at rawbw.com  Wed Jun 10 22:15:03 2009
From: yuri at rawbw.com (Yuri)
Date: Wed Jun 10 22:15:35 2009
Subject: 7.2-PRERELEASE hangs every time after running airodump-ng on ath0
	for a few hours
Message-ID: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>

I got this hang twice in the last day. After 2-3 hours of continuously  
run airodiump-ng (from net-mgmt/aircrack-ng) system stops responding  
to any mouse/keyboard clicks.

Why would aircrack-ng hang the system?
How to troubleshoot this?

I am current on ports and my kernel is from May 6.

Yuri


From glen.j.barber at gmail.com  Wed Jun 10 22:29:21 2009
From: glen.j.barber at gmail.com (Glen Barber)
Date: Wed Jun 10 22:29:27 2009
Subject: 7.2-PRERELEASE hangs every time after running airodump-ng on
	ath0 for a few hours
In-Reply-To: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>
References: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>
Message-ID: <4ad871310906101529n7886d189yab8b1bbdd174ba6b@mail.gmail.com>

Hi, Yuri

On Wed, Jun 10, 2009 at 6:15 PM, Yuri<yuri@rawbw.com> wrote:
> I got this hang twice in the last day. After 2-3 hours of continuously run
> airodiump-ng (from net-mgmt/aircrack-ng) system stops responding to any
> mouse/keyboard clicks.
>
> Why would aircrack-ng hang the system?
> How to troubleshoot this?
>
> I am current on ports and my kernel is from May 6.
>

By chance, are you running an out-of-sync kernel / userland?  You say
the kernel is from May 6, and in the subject is 7.2-PRERELEASE.  7.2
was -RELEASEd on May 3.

-- 
Glen Barber
From yuri at rawbw.com  Wed Jun 10 22:59:48 2009
From: yuri at rawbw.com (Yuri)
Date: Wed Jun 10 22:59:54 2009
Subject: 7.2-PRERELEASE hangs every time after running airodump-ng on
	ath0 for a few hours
In-Reply-To: <4ad871310906101529n7886d189yab8b1bbdd174ba6b@mail.gmail.com>
References: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>
	<4ad871310906101529n7886d189yab8b1bbdd174ba6b@mail.gmail.com>
Message-ID: <20090610155947.204348i5yvsi8x0k@webmail.rawbw.com>

Quoting "Glen Barber" <glen.j.barber@gmail.com>:

> By chance, are you running an out-of-sync kernel / userland?  You say
> the kernel is from May 6, and in the subject is 7.2-PRERELEASE.  7.2
> was -RELEASEd on May 3.

I am sorry, it's 7.2-RELEASE.

Yuri

From onemda at gmail.com  Wed Jun 10 23:47:16 2009
From: onemda at gmail.com (Paul B. Mahol)
Date: Wed Jun 10 23:47:22 2009
Subject: 7.2-PRERELEASE hangs every time after running airodump-ng on
	ath0 for a few hours
In-Reply-To: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>
References: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>
Message-ID: <3a142e750906101647r198aa62dncee8ba0a87c52597@mail.gmail.com>

On 6/10/09, Yuri <yuri@rawbw.com> wrote:
> I got this hang twice in the last day. After 2-3 hours of continuously
> run airodiump-ng (from net-mgmt/aircrack-ng) system stops responding
> to any mouse/keyboard clicks.
>
> Why would aircrack-ng hang the system?

Bugs in ath(4) and device itself can hang system.

-- 
Paul
From yuri at rawbw.com  Thu Jun 11 01:43:24 2009
From: yuri at rawbw.com (Yuri)
Date: Thu Jun 11 01:43:30 2009
Subject: 7.2-PRERELEASE hangs every time after running airodump-ng on
	ath0 for a few hours
In-Reply-To: <4ad871310906101529n7886d189yab8b1bbdd174ba6b@mail.gmail.com>
References: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>
	<4ad871310906101529n7886d189yab8b1bbdd174ba6b@mail.gmail.com>
Message-ID: <20090610184323.15063sewh2w3ua2o@webmail.rawbw.com>

Quoting "Glen Barber" <glen.j.barber@gmail.com>:

> By chance, are you running an out-of-sync kernel / userland?  You say
> the kernel is from May 6, and in the subject is 7.2-PRERELEASE.  7.2
> was -RELEASEd on May 3.

I just updated to more recent 72-STABLE and still got the hang.

Yuri

From rpaulo at gmail.com  Thu Jun 11 12:04:32 2009
From: rpaulo at gmail.com (Rui Paulo)
Date: Thu Jun 11 12:39:13 2009
Subject: 7.2-PRERELEASE hangs every time after running airodump-ng on
	ath0 for a few hours
In-Reply-To: <20090610184323.15063sewh2w3ua2o@webmail.rawbw.com>
References: <20090610151502.9206375pvaxw7pes@webmail.rawbw.com>
	<4ad871310906101529n7886d189yab8b1bbdd174ba6b@mail.gmail.com>
	<20090610184323.15063sewh2w3ua2o@webmail.rawbw.com>
Message-ID: <9793F82E-F48A-45F6-AC38-B7DE4BEA1D8E@gmail.com>


On 11 Jun 2009, at 02:43, Yuri wrote:

> Quoting "Glen Barber" <glen.j.barber@gmail.com>:
>
>> By chance, are you running an out-of-sync kernel / userland?  You say
>> the kernel is from May 6, and in the subject is 7.2-PRERELEASE.  7.2
>> was -RELEASEd on May 3.
>
> I just updated to more recent 72-STABLE and still got the hang.

It would help if you could provide the backtrace.

http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-gdb.html

Also, you may want to try HEAD because monitor mode has changed quite  
a bit in HEAD.

Regards,
--
Rui Paulo

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090611/9bdc27fc/PGP.pgp
From patfbsd at davenulle.org  Thu Jun 11 15:30:11 2009
From: patfbsd at davenulle.org (Patrick =?ISO-8859-15?Q?Lamaizi=E8re?=)
Date: Thu Jun 11 15:30:18 2009
Subject: Hardware random number generator
Message-ID: <20090611171345.113207f0@baby-jane.lamaiziere.net>

Hello,

There are two ways to managed a RNG in FreeBSD, I'm asking which is the
best:
- feed the random subsystem via random_harvest(9)
- add the RNG to random(4) as the Nehemiah RNG (VIA padlock)

In fact, i do not understand why the Nehemiah RNG is not managed by
padlock(4).

Thanks, regards.

From pawel.worach at gmail.com  Thu Jun 11 17:09:29 2009
From: pawel.worach at gmail.com (Pawel Worach)
Date: Thu Jun 11 17:09:37 2009
Subject: Clang: now available from a SVN server near you!
In-Reply-To: <20090609101745.GE48776@hoeg.nl>
References: <20090604093831.GE48776@hoeg.nl>
	<20090608.120552.756910862.imp@bsdimp.com>
	<d227e09e0906081346j3ee7822n24c1c4968088647d@mail.gmail.com>
	<20090609101745.GE48776@hoeg.nl>
Message-ID: <d227e09e0906111009j6bea1bfewd79aa75f6575c3fa@mail.gmail.com>

On Tue, Jun 9, 2009 at 12:17, Ed Schouten<ed@80386.nl> wrote:
> * Pawel Worach <pawel.worach@gmail.com> wrote:
>> So runtime performance is on par with gcc, code size is a bit bigger
>> so there is still room for optimization in LLVM.
>
> I don't agree on the code size. Code size is comparable. I just did a
> quick ls through /bin. There also seem to be a lot of cases where Clang
> generates smaller binaries.
>
> Some time ago the binaries were indeed a lot bigger, but that turned out
> to be a bug in Clang, where a compiler flag had a wrong default value,
> namely the flag that determined whether zero-initialized data had to go
> in BSS or not.

Here are some updated results with llvm/clang r73189.

Compile time and executable size for amd64 LINT (without profiling
because clang does not support it yet):

clang:
314.582u 29.555s 5:44.72 99.8%  14250+2428k 0+0io 481pf+0w

  text            data     bss     dec     hex filename
18034015        3563057 2910192 24507264        175f380 kernel
-rwxr-xr-x  1 root  wheel  27208927 Jun 11 14:34 kernel

gcc:
510.206u 30.260s 8:56.93 100.6% 7032+2414k 0+0io 14pf+0w

  text            data     bss     dec     hex filename
17342564        2623505 2918704 22884773        15d31a5 kernel
-rwxr-xr-x  1 root  wheel  24115620 Jun 11 14:45 kernel

Also an updated ports/benchmarks/libmicro run available here:
http://www.vlakno.cz/~pwo/lm/2009-06-11/lm.html

-- 
Pawel
From msnkipa at mail.ru  Fri Jun 12 21:55:46 2009
From: msnkipa at mail.ru (=?koi8-r?Q?=ED=C9=C8=C1=C9=CC_=EB=C9=D0=C1?=)
Date: Fri Jun 12 21:55:53 2009
Subject: GNAT on FreeBSD amd64
Message-ID: <E1MFEVh-00073Q-00.msnkipa-mail-ru@f261.mail.ru>

I need gnat on FreeBSD 7.2 amd64, but in ports it is only for i386 :(. I have tired to compile cross compiler under linux, and now I can compile my ada programs under linux for FreeBSD, but I can`t compile native amd64 FreeBSD gcc. Can anybody help me with GNAT under amd64 FreeBSD?
From dan.naumov at gmail.com  Fri Jun 12 22:07:41 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Fri Jun 12 22:18:45 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>
Message-ID: <cf9b1ee00906121507n24ec1f7dt89687ce2b85803b1@mail.gmail.com>

As promised, I took a go at this new BSDInstaller, I wrote down some
of my thoughts. Since I don't know if this is the kind of feedback you
are looking for, here is just a part of it. As you can probably guess
from it, I deal with usability issues in software applications a lot,
hence my point of view, but obviously I can find my way around a UNIX
without much issues.


FreeBSD-20090608-1522-8.0-CURRENT image was used:
=============================================
Initial BOOT:
The boot goes as planned, but the user is dropped off to a login
prompt. There is no direct indication for the user regarding how to
proceed. Adding a note just before the login prompt saying something
like "login as "root" with an empty password" would be nice. Again, as
soon the user does log on, there is no indication as to how to launch
the actual installer. This, as well as the previous problem could be
solved by presenting the user with a simple menu screen, like this:

Welcome to FreeBSD, choose your option:
1: Launch the FreeBSD installer
2: Drop to an emergency rescue shell

This would eleminate the actual need to log in as well, we could just
autologin as root for both options.

Initial options in the actual installer:
<Change Video Font> option: the names in a long list don't actually
tell the user anything regarding how does any of the fonts look. Would
it be possible to have a small sample preview next to each presented
font option?
<Change Screen Map> option: again, the screen map options do not
really explain the user what he should be choosing. A short
description of every option is needed at the very least.
<Change Keymap> option: ditto, same thing.

Another approach to this altogether could be a serious of questions
like: "What country do you live in? What language is your keyboard?"
providing multiple options for an answer for each question. After a
series of questions, the installer could just automatically pick up
most sane defaults based on the answers given by the user and finally
ask "Does the fonts on the screen look alright to you? Try typing in
here, do the expect characters appear as they should?" and if the user
is unsatisfied, he can be dropped to the kinds of a big selection list
which is used by the installer right now.

<Select Kernel Module to Load: Again, each option needs at least a
short description, a lot of the kernel module names are cryptic and a
new user cannot be reasonably expected to evaluate and make an
informed decision regarding what he does want and what he doesn't
based on a simple file list of the modules.

<Begin Installation> screen: The screen states that in order to do a
manual installation "<SNIP> login as root, and follow the instructions
given in the file /README". There is no indication regarding how the
user is supposed to open the README file, at least ONE sane option
should be suggested (for example: login as root and type "vi README"
at the command prompt to read the instructions regarding installing
FreeBSD manually.
=============================================


If you would like me to continue, please let me know.

Sincerely,
- Dan Naumov




On Tue, Jun 9, 2009 at 7:58 PM, Scott Ullrich<sullrich@gmail.com> wrote:
> On Tue, Jun 9, 2009 at 11:50 AM, Vincent Hoffman<vince@unsane.co.uk> wrote:
> [snip]
>>
>> That said, there have been a few projects to update/replace/whatever
>> sysinstall, look at the desktopBSD installer (bsdinstaller) and
>> finstall. I'm not sure what the status of either of these 2 are though.
>
> I was holding off on announcing this publicly but I guess now is a good time.
>
> Currently we are building FreeBSD-CURRENT + BSDInstaller snapshots
> every 12 hours or so at:
> http://snapshots.pfsense.org/FreeBSD_8_0/?C=M;O=D
>
> More information about BSDInstaller / FreeBSD integration is located
> here: http://wiki.freebsd.org/FreeBSD/BSDInstaller2009
>
> If folks would like to help out with the remaining needed items that
> would be awesome.
>
> Scott
>
From yuri at rawbw.com  Sat Jun 13 07:25:03 2009
From: yuri at rawbw.com (Yuri)
Date: Sat Jun 13 07:25:10 2009
Subject: top shows that multithreaded program uses 19623.14% CPU
Message-ID: <4A33544C.1070902@rawbw.com>

I ran then program with 1500 threads and in top it looked the this:

 PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    CPU COMMAND
20382 yuri       1500  99    0   641M   462M umtxn   0   0:00 19623.14% 
quicksort
<skipped further>

Where can I read how CPU column is calculated?

Some time ago I saw another weird fenomenon when CPU column: 5% CPU load 
for the pocess that just cycles in CPU.

Yuri

7.2-STABLE, i7-920 (4 CPUs)
From xorquewasp at googlemail.com  Sat Jun 13 10:16:53 2009
From: xorquewasp at googlemail.com (xorquewasp@googlemail.com)
Date: Sat Jun 13 10:16:59 2009
Subject: GNAT on FreeBSD amd64
In-Reply-To: <E1MFEVh-00073Q-00.msnkipa-mail-ru@f261.mail.ru>
References: <E1MFEVh-00073Q-00.msnkipa-mail-ru@f261.mail.ru>
Message-ID: <20090613101638.GA39139@logik.internal.network>

On 2009-06-13 01:42:05, ?????? ???? wrote:
> I need gnat on FreeBSD 7.2 amd64, but in ports it is only for i386 :(. I have
> tired to compile cross compiler under linux, and now I can compile my ada
> programs under linux for FreeBSD, but I can`t compile native amd64 FreeBSD gcc.
> Can anybody help me with GNAT under amd64 FreeBSD?

Hello.

A port is on the way, I'm working on it. I've also spoken to Karel Miklav, the
maintainer of the current lang/gnat-gcc ports with regards to updating them to
be amd64 compatible too.

I'm currently blocked waiting for a replacement for an RMA'd hard disk but
development will resume the minute it turns up.

Nice to see some interest in this port!
From freebsd at akruijff.dds.nl  Sat Jun 13 11:18:25 2009
From: freebsd at akruijff.dds.nl (Alex de Kruijff)
Date: Sat Jun 13 11:18:31 2009
Subject: Porting problem with gnu configure (c++ -V)
Message-ID: <20090613110618.GA1462@Alex1.lan>

I'm converting my port samesame to use gnu configue, but came a cross a
problem that is beond me. I'm able to run aclocal, autoconf, autoheader
and automake --add-missing -c. I've read the docs for autoconf and
automake and search the web, but dont know how to solve elegantly. I
would proberbly be able to hack configure, but prevere to edit only the
source files. Do other porters with more experiance have any tips for
me?

The machine runs on FreeBSD 6.1-p20.

When building the port I get this message:

===>  Configuring for samesame-1.3
configure: WARNING: you should use --build, --host, --target
checking for a BSD-compatible install... /usr/bin/install -c -o root -g
wheel
checking whether build environment is sane... yes
checking for gawk... no
checking for mawk... no
checking for nawk... nawk
checking whether make sets $(MAKE)... yes
checking for C++ compiler default output file name...
configure: error: C++ compiler cannot create executables
See `config.log' for more details.
===>  Script "configure" failed unexpectedly.


- Config.log show the following:
configure:2329: c++ --version >&5
c++ (GCC) 3.4.4 [FreeBSD] 20050518
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is
NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.

configure:2332: $? = 0
configure:2339: c++ -v >&5
Using built-in specs.   
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 3.4.4 [FreeBSD] 20050518
configure:2342: $? = 0
configure:2349: c++ -V >&5 <---------------------------- ------- -------
c++: `-V' option must have argument
configure:2352: $? = 1
configure:2375: checking for C++ compiler default output file name
configure:2402: c++ -O2 -fno-strict-aliasing -pipe -DWITH_DISK_STORAGE
-DWITH_MM
AP -D'TEMP_STORAGE_DIR="/tmp"' -DPATH_INIT=256
-DSTATIC_CACHE_CAPACITY=8192   co
nftest.cpp  >&5
<command line>:4:1: macro names must be identifiers
configure:2405: $? = 1
configure:2443: result:
configure: failed program was:
| /* confdefs.h.  */ /
| #define PACKAGE_NAME "SameSame"
| #define PACKAGE_TARNAME "samesame"
| #define PACKAGE_VERSION "1.3"
| #define PACKAGE_STRING "SameSame 1.3"
| #define PACKAGE_BUGREPORT "samesame@akruijff.dds.nl"
| #define PACKAGE "samesame"
| #define VERSION "1.3"
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:2449: error: C++ compiler cannot create executables
See `config.log' for more details.

-- 
Alex

Please copy the original recipients, otherwise I may not read your reply.

http://samesame.kruijff.org/

From freebsd at akruijff.dds.nl  Sat Jun 13 13:32:42 2009
From: freebsd at akruijff.dds.nl (Alex de Kruijff)
Date: Sat Jun 13 13:33:39 2009
Subject: Porting problem with gnu configure (c++ -V)
In-Reply-To: <20090613120117.GA28947@pm513-1.comsys.ntu-kpi.kiev.ua>
References: <20090613110618.GA1462@Alex1.lan>
	<20090613120117.GA28947@pm513-1.comsys.ntu-kpi.kiev.ua>
Message-ID: <20090613133238.GC1462@Alex1.lan>

On Sat, Jun 13, 2009 at 03:01:17PM +0300, Andrey Simonenko wrote:
> Hello,
> 
> On Sat, Jun 13, 2009 at 01:06:18PM +0200, Alex de Kruijff wrote:
> > I'm converting my port samesame to use gnu configue, but came a cross a
> > problem that is beond me. I'm able to run aclocal, autoconf, autoheader
> > and automake --add-missing -c. I've read the docs for autoconf and
> > automake and search the web, but dont know how to solve elegantly. I
> > would proberbly be able to hack configure, but prevere to edit only the
> > source files. Do other porters with more experiance have any tips for
> > me?
> 
> It's better to see your version for configure.ac, since without its
> content it is hard to say something.

Oke here it is:

# Process this file with autoconf to produce a configure script.

AC_PREREQ(2.61)
AC_INIT(FULL-PACKAGE-NAME, VERSION, BUG-REPORT-ADDRESS)
AM_INIT_AUTOMAKE
AC_CONFIG_SRCDIR([configure.h])
AC_CONFIG_HEADER([config.h])

# Checks for programs.
AC_PROG_CXX
#AC_PROG_CC
AC_PROG_INSTALL
AC_PROG_LN_S

# Checks for libraries.

# Checks for header files.
AC_HEADER_STDC
AC_CHECK_HEADERS([fcntl.h limits.h locale.h stddef.h stdint.h stdlib.h
string.h unistd.h])

# Checks for typedefs, structures, and compiler characteristics.
AC_C_CONST
AC_C_INLINE
AC_TYPE_OFF_T
AC_TYPE_PID_T
AC_TYPE_SIZE_T
AC_HEADER_TIME

# Checks for library functions.
AC_FUNC_LSTAT
AC_FUNC_LSTAT_FOLLOWS_SLASHED_SYMLINK
AC_FUNC_MALLOC
AC_FUNC_MEMCMP
AC_FUNC_MMAP
AC_TYPE_SIGNAL
AC_FUNC_STAT
AC_CHECK_FUNCS([gettimeofday localeconv memset mkdir munmap rmdir
strstr])

AC_CONFIG_FILES([Makefile])
AC_OUTPUT

-- 
Alex

Please copy the original recipients, otherwise I may not read your reply.

http://samesame.kruijff.org/

From msnkipa at mail.ru  Sat Jun 13 13:59:14 2009
From: msnkipa at mail.ru (=?koi8-r?Q?=ED=C9=C8=C1=C9=CC_=EB=C9=D0=C1?=)
Date: Sat Jun 13 13:59:21 2009
Subject: pthread_mutexattr_getprioceiling error?
Message-ID: <E1MFTlI-0008Sd-00.msnkipa-mail-ru@f60.mail.ru>

Next little program:
#include <pthread.h>
#include <iostream> 

int main()
{
        pthread_mutexattr_t t;
        if (pthread_mutexattr_init(&t)) return 1;
        int i;
        std::cout << pthread_mutexattr_getprioceiling(&t, &i) << std::endl;
}

always print 22. It means that pthread_mutexattr_getprioceiling always fails with EINVAL. Under Linux this example works fine, but under FreeBSD 7.2 it does`n work. Is it a bug in FreeBSD thread library or it ai my misunderstanding?
From doconnor at gsoft.com.au  Sat Jun 13 14:45:51 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Sat Jun 13 14:45:59 2009
Subject: Booting from FAT(32)
Message-ID: <200906140015.44560.doconnor@gsoft.com.au>

Does anyone have a recipe for such a thing?
I currently have a split FAT32/UFS install USB stick but I'd like to get 
it to just FAT32 if possible..

I think I can do it once I can run the loader - I can then load an MFS 
and use UZIP for the FSs and so on..

I have been playing with Syslinux but it doesn't like 
loading /boot/loader as a linux kernel and I'm not sure which format to 
try.

I think I can get it to work using the memdisk syslinux thing but that 
seems like a kludge..

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090613/93770427/attachment.pgp
From jilles at stack.nl  Sat Jun 13 15:35:09 2009
From: jilles at stack.nl (Jilles Tjoelker)
Date: Sat Jun 13 15:35:17 2009
Subject: pthread_mutexattr_getprioceiling error?
In-Reply-To: <E1MFTlI-0008Sd-00.msnkipa-mail-ru@f60.mail.ru>
References: <E1MFTlI-0008Sd-00.msnkipa-mail-ru@f60.mail.ru>
Message-ID: <20090613153441.GA35600@stack.nl>

On Sat, Jun 13, 2009 at 05:59:12PM +0400, ?????? ???? wrote:
> Next little program:
> #include <pthread.h>
> #include <iostream> 

> int main()
> {
>         pthread_mutexattr_t t;
>         if (pthread_mutexattr_init(&t)) return 1;
>         int i;
>         std::cout << pthread_mutexattr_getprioceiling(&t, &i) << std::endl;
> }

> always print 22. It means that pthread_mutexattr_getprioceiling always
> fails with EINVAL. Under Linux this example works fine, but under
> FreeBSD 7.2 it does`n work. Is it a bug in FreeBSD thread library or
> it ai my misunderstanding?

The priority ceiling is only meaningful with the PTHREAD_PRIO_PROTECT
protocol (pthread_mutexattr_setprotocol()). The FreeBSD threads library
returns EINVAL if you try to get/set the priority ceiling with another
protocol.

-- 
Jilles Tjoelker
From adrian at freebsd.org  Sat Jun 13 16:05:21 2009
From: adrian at freebsd.org (Adrian Chadd)
Date: Sat Jun 13 16:05:29 2009
Subject: Booting from FAT(32)
In-Reply-To: <200906140015.44560.doconnor@gsoft.com.au>
References: <200906140015.44560.doconnor@gsoft.com.au>
Message-ID: <d763ac660906130833h1043b1eat3b83964bc81bcbea@mail.gmail.com>

I believe there's some NetBSD FAT32 aware loader. Dianora stumbled
across it when I was talking about this very thing earlier.

I've been meaning to trial it out actually. It sure would be
interesting to boot a kernel/mfs off of a DOS bootable FAT USB.



Adrian

2009/6/13 Daniel O'Connor <doconnor@gsoft.com.au>:
> Does anyone have a recipe for such a thing?
> I currently have a split FAT32/UFS install USB stick but I'd like to get
> it to just FAT32 if possible..
>
> I think I can do it once I can run the loader - I can then load an MFS
> and use UZIP for the FSs and so on..
>
> I have been playing with Syslinux but it doesn't like
> loading /boot/loader as a linux kernel and I'm not sure which format to
> try.
>
> I think I can get it to work using the memdisk syslinux thing but that
> seems like a kludge..
>
> --
> Daniel O'Connor software and network engineer
> for Genesis Software - http://www.gsoft.com.au
> "The nice thing about standards is that there
> are so many of them to choose from."
> ?-- Andrew Tanenbaum
> GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
>
From deischen at freebsd.org  Sat Jun 13 16:08:36 2009
From: deischen at freebsd.org (Daniel Eischen)
Date: Sat Jun 13 16:08:43 2009
Subject: pthread_mutexattr_getprioceiling error?
In-Reply-To: <E1MFTlI-0008Sd-00.msnkipa-mail-ru@f60.mail.ru>
References: <E1MFTlI-0008Sd-00.msnkipa-mail-ru@f60.mail.ru>
Message-ID: <Pine.GSO.4.64.0906131158130.7673@sea.ntplx.net>

On Sat, 13 Jun 2009, ?????? ???? wrote:

> Next little program:
> #include <pthread.h>
> #include <iostream>
>
> int main()
> {
>        pthread_mutexattr_t t;
>        if (pthread_mutexattr_init(&t)) return 1;
>        int i;
>        std::cout << pthread_mutexattr_getprioceiling(&t, &i) << std::endl;
> }
>
> always print 22. It means that pthread_mutexattr_getprioceiling always 
> fails with EINVAL. Under Linux this example works fine, but under 
> FreeBSD 7.2 it does`n work. Is it a bug in FreeBSD thread library or 
> it ai my misunderstanding?

Well, you can't get the priority ceiling of a _mutex_ if
it the protocol is PTHREAD_PRIO_NONE.  See the POSIX spec,
excerpt here (pthread_mutex_getprioceiling):

     These functions shall fail if:

     [EINVAL]
         The protocol attribute of mutex is PTHREAD_PRIO_NONE.

POSIX makes no mention of this for operations on a mutex attribute,
but I have no idea why it wouldn't have the same error returned
under similar conditions.

Why would you want to get the priority ceiling of a mutex
attribute that is not PTHREAD_PRIO_PROTECT?  What value
could the implementation return in this case anyway, it
really doesn't make sense?

-- 
DE
From dnelson at allantgroup.com  Sat Jun 13 18:43:09 2009
From: dnelson at allantgroup.com (Dan Nelson)
Date: Sat Jun 13 18:43:15 2009
Subject: top shows that multithreaded program uses 19623.14% CPU
In-Reply-To: <4A33544C.1070902@rawbw.com>
References: <4A33544C.1070902@rawbw.com>
Message-ID: <20090613184303.GK56070@dan.emsphone.com>

In the last episode (Jun 13), Yuri said:
> I ran then program with 1500 threads and in top it looked the this:
> 
>  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    CPU COMMAND
> 20382 yuri       1500  99    0   641M   462M umtxn   0   0:00 19623.14%  quicksort
> <skipped further>
> 
> Where can I read how CPU column is calculated?
> 
> Some time ago I saw another weird fenomenon when CPU column: 5% CPU load 
> for the pocess that just cycles in CPU.

Top doesn't seem to show the right %CPU at all for threaded apps unless you
hit H to view each thread individually.  Then you get accurate numbers (but
you have to manually sum up the threads usage if you want to see the total
%CPU for an entire process).

-- 
	Dan Nelson
	dnelson@allantgroup.com
From sullrich at gmail.com  Sat Jun 13 19:47:23 2009
From: sullrich at gmail.com (Scott Ullrich)
Date: Sat Jun 13 19:47:36 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906121507n24ec1f7dt89687ce2b85803b1@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com> 
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl> 
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com> 
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com> 
	<cf9b1ee00906121507n24ec1f7dt89687ce2b85803b1@mail.gmail.com>
Message-ID: <d5992baf0906131247s4c1d64aembf826f6d8d2161ef@mail.gmail.com>

On Fri, Jun 12, 2009 at 6:07 PM, Dan Naumov<dan.naumov@gmail.com> wrote:
> As promised, I took a go at this new BSDInstaller, I wrote down some
> of my thoughts. Since I don't know if this is the kind of feedback you
> are looking for, here is just a part of it. As you can probably guess
> from it, I deal with usability issues in software applications a lot,
> hence my point of view, but obviously I can find my way around a UNIX
> without much issues.

Thanks!!

>
> FreeBSD-20090608-1522-8.0-CURRENT image was used:
> =============================================
> Initial BOOT:
> The boot goes as planned, but the user is dropped off to a login
> prompt. There is no direct indication for the user regarding how to
> proceed. Adding a note just before the login prompt saying something
> like "login as "root" with an empty password" would be nice. Again, as
> soon the user does log on, there is no indication as to how to launch
> the actual installer. This, as well as the previous problem could be
> solved by presenting the user with a simple menu screen, like this:
>
> Welcome to FreeBSD, choose your option:
> 1: Launch the FreeBSD installer
> 2: Drop to an emergency rescue shell

That is one part that I have not touched on as of yet.  DragonFly has
a "installer" account that launches the BSDI but I am not too sure if
thats how we should visit this or not?

> This would eleminate the actual need to log in as well, we could just
> autologin as root for both options.
>
> Initial options in the actual installer:
> <Change Video Font> option: the names in a long list don't actually
> tell the user anything regarding how does any of the fonts look. Would
> it be possible to have a small sample preview next to each presented
> font option?
> <Change Screen Map> option: again, the screen map options do not
> really explain the user what he should be choosing. A short
> description of every option is needed at the very least.
> <Change Keymap> option: ditto, same thing.

Good ideas.  Any chance you can add them to the wiki?

> Another approach to this altogether could be a serious of questions
> like: "What country do you live in? What language is your keyboard?"
> providing multiple options for an answer for each question. After a
> series of questions, the installer could just automatically pick up
> most sane defaults based on the answers given by the user and finally
> ask "Does the fonts on the screen look alright to you? Try typing in
> here, do the expect characters appear as they should?" and if the user
> is unsatisfied, he can be dropped to the kinds of a big selection list
> which is used by the installer right now.
>
> <Select Kernel Module to Load: Again, each option needs at least a
> short description, a lot of the kernel module names are cryptic and a
> new user cannot be reasonably expected to evaluate and make an
> informed decision regarding what he does want and what he doesn't
> based on a simple file list of the modules.

Also a good idea and should be relatively easy if we can pull the
short description dynamically somehow vs. hard coding it?

> <Begin Installation> screen: The screen states that in order to do a
> manual installation "<SNIP> login as root, and follow the instructions
> given in the file /README". There is no indication regarding how the
> user is supposed to open the README file, at least ONE sane option
> should be suggested (for example: login as root and type "vi README"
> at the command prompt to read the instructions regarding installing
> FreeBSD manually.

This is leftover and also needs to be addressed.   So far I have
focused on getting the BSDI ported to FreeBSD and to remove the
"pfSense'sms" that where present.

Really appreciate all of the advice but I am hoping that it does not
get lost.   Do you mind helping me update the Wiki with your
suggestions?   Maybe add a usability improvements section.   If not I
can get it updated later this weekend.

Also I am working on getting 7-STABLE snapshots together and embedded
images suitable for compact flash cards ala DD/physdiskwrite.exe.

Scott
From sullrich at gmail.com  Sat Jun 13 20:08:02 2009
From: sullrich at gmail.com (Scott Ullrich)
Date: Sat Jun 13 20:08:14 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906131256y6aa51a4fp5b921dbc80472a15@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com> 
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl> 
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com> 
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com> 
	<cf9b1ee00906121507n24ec1f7dt89687ce2b85803b1@mail.gmail.com> 
	<d5992baf0906131247s4c1d64aembf826f6d8d2161ef@mail.gmail.com> 
	<cf9b1ee00906131256y6aa51a4fp5b921dbc80472a15@mail.gmail.com>
Message-ID: <d5992baf0906131307p4042e951i493d0d3a1f656137@mail.gmail.com>

On Sat, Jun 13, 2009 at 3:56 PM, Dan Naumov<dan.naumov@gmail.com> wrote:
> Which Wiki do you want me to contribute this to?
> http://wiki.freebsd.org/FreeBSD/BSDInstaller2009 or
> http://wiki.bsdinstaller.org/wikka.php?wakka=BSDInstaller ? Whichever
> it is, I am not that experienced with editing Wikis, so perhaps you
> could create the needed sections/subsections for usability issues and
> I would look into filling my thoughts into them when I have some time?

I just added a section to
http://wiki.freebsd.org/FreeBSD/BSDInstaller2009 ..  It's quite easy
to work the wiki.

Thanks for the suggestions and testing the Installer!

Scott
From des at des.no  Sat Jun 13 20:23:52 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Sat Jun 13 20:24:08 2009
Subject: Porting problem with gnu configure (c++ -V)
In-Reply-To: <20090613133238.GC1462@Alex1.lan> (Alex de Kruijff's message of
	"Sat, 13 Jun 2009 15:32:38 +0200")
References: <20090613110618.GA1462@Alex1.lan>
	<20090613120117.GA28947@pm513-1.comsys.ntu-kpi.kiev.ua>
	<20090613133238.GC1462@Alex1.lan>
Message-ID: <86prd7x4q1.fsf@ds4.des.no>

Alex de Kruijff <freebsd@akruijff.dds.nl> writes:
> Andrey Simonenko <simon@comsys.ntu-kpi.kiev.ua> writes:
> > It's better to see your version for configure.ac, since without its
> > content it is hard to say something.
> Oke here it is:

This configure.ac can not possibly have produced the configure.log you
posted earlier.  Can you please show us the *real* configure.ac and / or
configure.log?

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From dan.naumov at gmail.com  Sat Jun 13 19:56:25 2009
From: dan.naumov at gmail.com (Dan Naumov)
Date: Sat Jun 13 20:58:27 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <d5992baf0906131247s4c1d64aembf826f6d8d2161ef@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>
	<cf9b1ee00906121507n24ec1f7dt89687ce2b85803b1@mail.gmail.com>
	<d5992baf0906131247s4c1d64aembf826f6d8d2161ef@mail.gmail.com>
Message-ID: <cf9b1ee00906131256y6aa51a4fp5b921dbc80472a15@mail.gmail.com>

Which Wiki do you want me to contribute this to?
http://wiki.freebsd.org/FreeBSD/BSDInstaller2009 or
http://wiki.bsdinstaller.org/wikka.php?wakka=BSDInstaller ? Whichever
it is, I am not that experienced with editing Wikis, so perhaps you
could create the needed sections/subsections for usability issues and
I would look into filling my thoughts into them when I have some time?


Sincerely,
- Dan Naumov


On Sat, Jun 13, 2009 at 10:47 PM, Scott Ullrich<sullrich@gmail.com> wrote:
> Really appreciate all of the advice but I am hoping that it does not
> get lost. ? Do you mind helping me update the Wiki with your
> suggestions? ? Maybe add a usability improvements section. ? If not I
> can get it updated later this weekend.
>
> Also I am working on getting 7-STABLE snapshots together and embedded
> images suitable for compact flash cards ala DD/physdiskwrite.exe.
>
> Scott
>
From jgreco at ns.sol.net  Sat Jun 13 21:54:17 2009
From: jgreco at ns.sol.net (Joe Greco)
Date: Sat Jun 13 21:54:24 2009
Subject: Maybe confused about AMD64 / i386 compatibility
Message-ID: <200906132055.n5DKtTAW063495@aurora.sol.net>

Background (most probably not necessary):

We use rsync snapshots and cheap IDE disks as part of our backup strategy.
Recently, a backup server started getting near full, and with 1.5TB disks
being what they are, we shot to replace a 750GB IDE with a 1.5TB SATA.

The backup server was upgraded to FreeBSD 7.2R/i386, it sports a 4GB flash
which leaves two drive bays in the Intel ISP1100 chassis available. 
Adding a SIL3112A gives us the SATA.  Good to go.  Create 1.5TB filesys,
and copy the old 750GB to it.

Option 1, using dd followed by growfs, doesn't work.  Groan.  growfs bombs
out with a negative integer in some array, and it isn't clear to me what
is happening.  So let's do the old slow but reliable dump/restore.

Except that a dump/restore of the filesystem, which is full of hardlinks,
causes massive resource consumption and eventual exhaustion.  Okay, fine.

Option 2, dump with gzip -1 to a file on the new drive (it'll have enough
space to restore in a separate step).  This works great.  Verify file
integrity (~400GB).  Also just fine.

Restore.  Watch restore grow to near 3GB and then bail.  Okay, obviously
an i386 architecture won't handle this restore.  So, think, aha, I'll go
and put it on a FreeBSD 7.1R/amd64 Opteron box with 8GB that is idle at
the moment.

-->> And here's where it all goes wonky.

Using a USB-to-SATA adapter, I plug in the drive to the amd64.  It sees
it.  I mount it.  All looks fine.  Good!

	# mount /dev/da1s1d /mnt
	# df
	Filesystem  1K-blocks       Used     Avail Capacity  Mounted on
	[...]
	/dev/da1s1d 1419039310 432472500 873043666    33%    /mnt

Now I try a restore.  So I do "gunzip < ad3s1e.dmp2 | restore rvf -" ...
and gzip claims the data isn't compressed.

	# gunzip < ad3s1e.dmp2 | restore rvf -
	gunzip: stdin: not in gzip format
	Verify tape and initialize maps
	End-of-tape encountered
	Tape is not a dump tape

Look at the data, looks generally compressed.  But file(1) sees it
as "data", not gzip data.

So, scratch head, wonder if it might be some odd gzip incompatibility,
get latest GNU gzip, still no go.  Rather annoyed, I think, "but I know
I verified this was a good file" - so I take the disk back to the i386,
put it in on the SATA controller, and it works fine.

So suddenly I think the USB-to-SATA is garbage and is corrupting the
data.  I md5 the first portions of the file, and go back, put the disk
on the amd64 with the USB-to-SATA.

	# dd if=ad3s1e.dmp2 bs=2048 count=1 | md5 

Returns f86ea62b2c77c58691001371c283c7a0 on i386,
	43ca53ad03650bc8c8fd1279ce19a675 on amd64.

!!!!!!!

This seems really odd.  So just to be sure, I go back to the i386 box,
this time with the USB-to-SATA...  and it returns the correct answer.

Double "!!!!!!!!!"

Now I finally get smart and try working with the raw disk device. 

	# dd if=/dev/da1s1 bs=2048 count=1024 | md5
	1024+0 records in
	1024+0 records out
	2097152 bytes transferred in 7.189189 secs (291709 bytes/sec)
	504efcdd93164c496c07603ba9b3774c

In all cases:

7.1amd64/USB:	504efcdd93164c496c07603ba9b3774c
7.2i386/SATA:	504efcdd93164c496c07603ba9b3774c
7.1amd64/USB:	504efcdd93164c496c07603ba9b3774c

So the underlying hardware and driver support "seems" to be fine.

And now this brings me to my question.

In my experience, FreeBSD filesystems are portable across i386-amd64.

Are there circumstances where this isn't the case?

I'm happy to concede that the filesystem in question is rather largish
(1.5TB) and probably stresses things a little bit.

# ls -al
total 432471268
drwxr-xr-x    5 root  wheel              512 Jun 13 11:07 .
drwxr-xr-x   24 root  wheel              512 Feb 18 09:18 ..
drwxrwxr-x    2 root  operator           512 Jun  5 08:20 .snap
-rw-r--r--    1 root  wheel     434044319653 Jun 10 19:25 ad3s1e.dmp2
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c1
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c1024
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c256
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c4
-rw-r--r--    1 root  wheel               80 Jun 13 10:43 logfile
drwx------  293 root  wheel            18432 Jun  8 04:25 lost+found
-rw-rw-rw-    1 root  wheel       8589934592 Jun  8 00:33 swapfile
drwxrwxrwx    9 root  wheel              512 Jun  8 14:14 x

Oh, and just to add some pain and agony to the puzzle ... it all seems
to work fine on 6.1-RELEASE/amd64 ... and 7.0-RELEASE/amd64.  But on a
different machine.  Which is where I'm doing my extraction.

So I'm not sure what to make of this.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
From freebsd at akruijff.dds.nl  Sun Jun 14 00:13:39 2009
From: freebsd at akruijff.dds.nl (Alex de Kruijff)
Date: Sun Jun 14 00:13:45 2009
Subject: Porting problem with gnu configure (c++ -V)
In-Reply-To: <86prd7x4q1.fsf@ds4.des.no>
References: <20090613110618.GA1462@Alex1.lan>
	<20090613120117.GA28947@pm513-1.comsys.ntu-kpi.kiev.ua>
	<20090613133238.GC1462@Alex1.lan> <86prd7x4q1.fsf@ds4.des.no>
Message-ID: <20090614001336.GG1462@Alex1.lan>

On Sat, Jun 13, 2009 at 10:23:50PM +0200, Dag-Erling Sm??rgrav wrote:
> Alex de Kruijff <freebsd@akruijff.dds.nl> writes:
> > Andrey Simonenko <simon@comsys.ntu-kpi.kiev.ua> writes:
> > > It's better to see your version for configure.ac, since without its
> > > content it is hard to say something.
> > Oke here it is:
> 
> This configure.ac can not possibly have produced the configure.log you
> posted earlier.  Can you please show us the *real* configure.ac and / or
> configure.log?
> 
> DES
> -- 
> Dag-Erling Sm??rgrav - des@des.no

The file is processed by sed, but the only difference is:
-AC_INIT(FULL-PACKAGE-NAME, VERSION, BUG-REPORT-ADDRESS)
+AC_INIT(SameSame, 1.3, samesame@akruijff.dds.nl)

There is no configure.log.

-- 
Alex

Please copy the original recipients, otherwise I may not read your reply.

http://samesame.kruijff.org/

From jgreco at ns.sol.net  Sun Jun 14 00:44:41 2009
From: jgreco at ns.sol.net (Joe Greco)
Date: Sun Jun 14 00:44:48 2009
Subject: Maybe confused about AMD64 / i386 compatibility
In-Reply-To: <20090613230749.GA73896@server.vk2pj.dyndns.org> from "Peter
	Jeremy" at Jun 14, 2009 09:07:50 AM
Message-ID: <200906140044.n5E0iqYN072978@aurora.sol.net>

> On 2009-Jun-13 15:55:29 -0500, Joe Greco <jgreco@ns.sol.net> wrote:
> >Adding a SIL3112A gives us the SATA.
> 
> These are known to cause data corruption (check the archives).  I
> wouldn't trust anything that has passed through a SIL chip without
> independent validation.

I already said it had been validated.  gunzip|restore tvf was happy
with the entire thing.  A FreeBSD 6.1R/amd64 box is currently happily
RESTORING the SIL-written gzip'ed file using a USB-to-SATA adaptor,
TOO, so all evidence is that the on-disk file data is sane.

I checked for general data corruption at the device level with md5
and posted a brief summary of those results; the results are that
everything appears to be reading the disk blocks sanely.  That was
why I posted such a long summary of what had been done; I felt certain
someone would try to claim dodgy hardware.  The SIL does seem to
spit off lots of spurious interrupts, and does not work at all with
non-native SATA drives; being a backup system, I had already stress
tested various combinations of things, and I'm aware of the various
PC hardware deficiencies.

So, let me re-summarize and simplify the issue to clarify:

I have a large (~400GB) file on a large (~1.5TB) filesystem created
on 7.2R-i386.

7.2R-i386 reads the file correctly (via SIL or via USB-to-SATA).

6.1R-amd64 reads the file correctly (via USB-to-SATA).

7.1R-amd64 does NOT read the file correctly (via USB-to-SATA).

In all of the above cases, the underlying hardware and device drivers
appear to be returning the same data, as evidenced by dd <rawdev>|md5
of random portions of the disk.

In other words, the SIL is not in the equation.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
From peterjeremy at optushome.com.au  Sun Jun 14 01:03:13 2009
From: peterjeremy at optushome.com.au (Peter Jeremy)
Date: Sun Jun 14 01:03:20 2009
Subject: Maybe confused about AMD64 / i386 compatibility
In-Reply-To: <200906132055.n5DKtTAW063495@aurora.sol.net>
References: <200906132055.n5DKtTAW063495@aurora.sol.net>
Message-ID: <20090613230749.GA73896@server.vk2pj.dyndns.org>

On 2009-Jun-13 15:55:29 -0500, Joe Greco <jgreco@ns.sol.net> wrote:
>Adding a SIL3112A gives us the SATA.

These are known to cause data corruption (check the archives).  I
wouldn't trust anything that has passed through a SIL chip without
independent validation.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090614/a94f7850/attachment.pgp
From doconnor at gsoft.com.au  Sun Jun 14 01:04:11 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Sun Jun 14 01:04:18 2009
Subject: Booting from FAT(32)
In-Reply-To: <d763ac660906130833h1043b1eat3b83964bc81bcbea@mail.gmail.com>
References: <200906140015.44560.doconnor@gsoft.com.au>
	<d763ac660906130833h1043b1eat3b83964bc81bcbea@mail.gmail.com>
Message-ID: <200906141034.01890.doconnor@gsoft.com.au>

On Sun, 14 Jun 2009, Adrian Chadd wrote:
> I believe there's some NetBSD FAT32 aware loader. Dianora stumbled
> across it when I was talking about this very thing earlier.

I thought the FreeBSD loader spoke FAT32 already, although now I go and 
have a proper look I see otherwise :(


> I've been meaning to trial it out actually. It sure would be
> interesting to boot a kernel/mfs off of a DOS bootable FAT USB.

I think you can do it if you makefs /boot and feed it to syslinux using 
memdisk (I plan on trying this today).

I was thinking the loader had FAT support though so I could just get 
Syslinux to load the loader then it would do the rest. Although heck 
how hard can read only FAT support be to write? :)

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090614/179d9898/attachment.pgp
From doconnor at gsoft.com.au  Sun Jun 14 04:40:20 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Sun Jun 14 04:40:27 2009
Subject: Booting from FAT(32)
In-Reply-To: <200906141034.01890.doconnor@gsoft.com.au>
References: <200906140015.44560.doconnor@gsoft.com.au>
	<d763ac660906130833h1043b1eat3b83964bc81bcbea@mail.gmail.com>
	<200906141034.01890.doconnor@gsoft.com.au>
Message-ID: <200906141410.14771.doconnor@gsoft.com.au>

On Sun, 14 Jun 2009, Daniel O'Connor wrote:
> > I've been meaning to trial it out actually. It sure would be
> > interesting to boot a kernel/mfs off of a DOS bootable FAT USB.
>
> I think you can do it if you makefs /boot and feed it to syslinux
> using memdisk (I plan on trying this today).

I used Luigi's iso2flash.sh script (the guts thereof) and it worked fine, ie..
mkdir /tmp/boot
cp -r /boot /tmp/boot
[ fixup /tmp/boot/boot how you like ]
makefs -t ffs -o bsize=4096 -o fsize=512 -f 50 /tmp/boot.img /tmp/boot
bsdlabel -Bw -f /tmp/boot.img auto
bsdlabel -f /tmp/boot.img | sed -e '/  c:/{p;s/c:/a:/;}' | bsdlabel -R -f /tmp/boot.img /dev/stdin
gzip /tmp/boot.img

(Using -B obviated the need for the dd magic in the original script)

Copy /tmp/boot.img.gz to the USB stick and then you can use a syslinux conf 
like..
label FreeBSD
kernel memdisk
append initrd=boot.img.gz

I got the kernel to mount root off the FAT32 partition although you need to
make sure /dev exists and all the binaries & libs it wants are present.
(eg mkdir /mnt/dev ; cp -r /bin /sbin /lib /libexec /mnt)

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090614/752e1cb9/attachment.pgp
From kris at FreeBSD.org  Sun Jun 14 12:39:41 2009
From: kris at FreeBSD.org (Kris Kennaway)
Date: Sun Jun 14 12:40:03 2009
Subject: ACPI-fast default timecounter, but HPET 83% faster
In-Reply-To: <200904300846.41576.jhb@freebsd.org>
References: <200904270150.31912.pieter@degoeje.nl>	<7d6fde3d0904261927s1a67cf85jc982c1a68e30e081@mail.gmail.com>
	<200904300846.41576.jhb@freebsd.org>
Message-ID: <4A34EF89.10107@FreeBSD.org>

John Baldwin wrote:
> On Sunday 26 April 2009 10:27:42 pm Garrett Cooper wrote:
>> I'm seeing similar results.
>>
>> [root@orangebox /usr/home/gcooper]# dmesg | grep 'Timecounter "'
>> Timecounter "i8254" frequency 1193182 Hz quality 0
>> Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
>> Timecounter "HPET" frequency 14318180 Hz quality 900
>> [root@orangebox /usr/home/gcooper]# ./cgt
>> 1369355
>> [root@orangebox /usr/home/gcooper]# sysctl
>> kern.timecounter.hardware="ACPI-fast"
>> kern.timecounter.hardware: HPET -> ACPI-fast
>> [root@orangebox /usr/home/gcooper]# ./cgt
>> 772289
>>
>> Why's the default ACPI-fast? For power-saving functionality or because
>> of the `quality' factor? What is the criteria that determines the
>> `quality' of a clock as what's being reported above (I know what
>> determines the quality of a clock visually from a oscilloscope =])?
> 
> I suspect that the quality of the HPET driver is lower simply because no one
> had measured it previously and HPET is newer and less "proven".
> 

 From memory, HPET was massively slower on some of the AMD test hardware 
I was using.  There was a thread about it on one of the mailing lists, 
but I can't find it right now.

Kris
From Trond.Endrestol at fagskolen.gjovik.no  Sun Jun 14 15:44:42 2009
From: Trond.Endrestol at fagskolen.gjovik.no (=?ISO-8859-1?Q?Trond_Endrest=F8l?=)
Date: Sun Jun 14 15:44:55 2009
Subject: sysinstall, GJOURNAL and ZFS
In-Reply-To: <cf9b1ee00906121507n24ec1f7dt89687ce2b85803b1@mail.gmail.com>
References: <cf9b1ee00906090242h6dea1182h8e743b7ceec36c2c@mail.gmail.com>
	<alpine.BSF.2.00.0906091632430.6551@wojtek.tensor.gdynia.pl>
	<cf9b1ee00906090757v7d589dfch978076a97be724a9@mail.gmail.com>
	<4A2E84DC.1010900@unsane.co.uk>
	<d5992baf0906090958r250ae0afibd2c4320630778ba@mail.gmail.com>
	<cf9b1ee00906121507n24ec1f7dt89687ce2b85803b1@mail.gmail.com>
Message-ID: <20090614172116.C55547@ramstind.fig.ol.no>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, 13 Jun 2009 01:07+0300, Dan Naumov wrote:

> <Begin Installation> screen: The screen states that in order to do a
> manual installation "<SNIP> login as root, and follow the instructions
> given in the file /README". There is no indication regarding how the
> user is supposed to open the README file, at least ONE sane option
> should be suggested (for example: login as root and type "vi README"
> at the command prompt to read the instructions regarding installing
> FreeBSD manually.

Although longer to type, "less /README" would be a better alternative 
than using vi to view the contents of the file. You could otherwise 
risk the user destroying the README file due to unfamiliarity with 
this particular editor.

OTOH, why not adopt OpenBSD's afterboot(8) man page?

===

Almost completely off-topic:

An /altroot mechanism would be nice to have in FreeBSD. ;-)

There are at least two schools of thought regarding /altroot.

One school of thought would be to simply dd the contents of the root 
partition on to the altroot partition and run fsck on /altroot, 
assuming /root and /altroot have the same size. I believe this is how 
OpenBSD handles /altroot when configured to do so.

My school of thought, given a system with more than one hard drive, 
physical or virtual (as in RAID volumes), is to have /altroot appear 
as a regular root file system (partition a) on the other drive and 
thus being able to (manually) boot from the available root file 
system. Both hard drives should be made bootable with the EasyBoot 
loader placed in the MBRs and with regular boot blocks in the FreeBSD 
slices. Of course, /altroot should normally remain unmounted.

There is also the possible need of having /etc/fstab files with 
specific contents for each root file system. For instance, /'s 
/altroot should be mounted as /altroot's /. I'm not sure how to handle 
this scenario with either installers as you probably need to perform 
the installation twice, that's how I did it on a couple of recently 
installed systems, not to mention how to handle make 
install{kernel,world} for both of the two root file systems.


Just some random thoughts,
Trond.

- -- 
- ----------------------------------------------------------------------
Trond Endrest?l                  | Trond.Endrestol@fagskolen.gjovik.no
ACM, NAS, NUUG, SAGE, USENIX     |      FreeBSD 6.2-STABLE & Pine 4.64

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (FreeBSD)

iD8DBQFKNRrfbYWZalUoElsRAnzTAJ0StVu1t2FY1rr9JjI2MVQ0jexsAgCfZICv
kbpbqm4Vzv/vGYVTZEPwmfw=
=dPhW
-----END PGP SIGNATURE-----
From des at des.no  Sun Jun 14 22:49:14 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Sun Jun 14 22:49:22 2009
Subject: Porting problem with gnu configure (c++ -V)
In-Reply-To: <20090614001336.GG1462@Alex1.lan> (Alex de Kruijff's message of
	"Sun, 14 Jun 2009 02:13:36 +0200")
References: <20090613110618.GA1462@Alex1.lan>
	<20090613120117.GA28947@pm513-1.comsys.ntu-kpi.kiev.ua>
	<20090613133238.GC1462@Alex1.lan> <86prd7x4q1.fsf@ds4.des.no>
	<20090614001336.GG1462@Alex1.lan>
Message-ID: <86k53ev3br.fsf@ds4.des.no>

Alex de Kruijff <freebsd@akruijff.dds.nl> writes:
> There is no configure.log.

There is a config.log which you posted, but it was corrupted by your MUA.

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From malathiramya at gmail.com  Mon Jun 15 06:14:09 2009
From: malathiramya at gmail.com (malathi selvaraj)
Date: Mon Jun 15 06:14:39 2009
Subject: good morning to all
Message-ID: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>

i am new one to  freeBSD, kindly guide me

-- 
S.MALATHI
From ed at 80386.nl  Mon Jun 15 06:36:39 2009
From: ed at 80386.nl (Ed Schouten)
Date: Mon Jun 15 06:36:50 2009
Subject: good morning to all
In-Reply-To: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>
References: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>
Message-ID: <20090615063638.GS48776@hoeg.nl>

* malathi selvaraj <malathiramya@gmail.com> wrote:
> i am new one to  freeBSD, kindly guide me

Sure. In order to be eligible to use FreeBSD, you must buy a license for
$ 100,-. I expect this money to be transferred to my bank account as
soon as possible. The IBAN number is:

	NL30ABNA0385823426

-- 
 Ed Schouten <ed@80386.nl>
 WWW: http://80386.nl/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090615/fa395c78/attachment.pgp
From tim at clewlow.org  Mon Jun 15 06:53:10 2009
From: tim at clewlow.org (Tim Clewlow)
Date: Mon Jun 15 06:53:21 2009
Subject: good morning to all
In-Reply-To: <20090615063638.GS48776@hoeg.nl>
References: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>
	<20090615063638.GS48776@hoeg.nl>
Message-ID: <66423c5de580a711d0e9e9f8a6034a8a.squirrel@192.168.1.100>


> * malathi selvaraj <malathiramya@gmail.com> wrote:
>> i am new one to  freeBSD, kindly guide me
>
> Sure. In order to be eligible to use FreeBSD, you must buy a license
> for
> $ 100,-. I expect this money to be transferred to my bank account as
> soon as possible. The IBAN number is:
>
> 	NL30ABNA0385823426
>

You may also register at the secret special mailing list
iwannalottaspam.com

From william at futurecis.com  Mon Jun 15 06:56:24 2009
From: william at futurecis.com (william@futurecis.com)
Date: Mon Jun 15 06:56:32 2009
Subject: good morning to all
Message-ID: <1791695605-1245047251-cardhu_decombobulator_blackberry.rim.net-1998550347-@bxe1309.bisx.prod.on.blackberry>

I have honestly found the best guide/help to be the actual handbook. Unlike most open source projects with poor documentation; this is one of the best. Here is the link:

http://www.freebsd.org/doc/en/books/handbook/

-Will
------Original Message------
From: malathi selvaraj
Sender: owner-freebsd-hackers@freebsd.org
To: freebsd-hackers@freebsd.org
Subject: good morning to all
Sent: Jun 15, 2009 01:52

i am new one to  freeBSD, kindly guide me

-- 
S.MALATHI
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"


Sent from my BlackBerry? wireless device
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun 15 07:59:00 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun 15 07:59:07 2009
Subject: good morning to all
In-Reply-To: <20090615063638.GS48776@hoeg.nl>
References: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>
	<20090615063638.GS48776@hoeg.nl>
Message-ID: <alpine.BSF.2.00.0906150957250.17105@wojtek.tensor.gdynia.pl>

> * malathi selvaraj <malathiramya@gmail.com> wrote:
>> i am new one to  freeBSD, kindly guide me
>
> Sure. In order to be eligible to use FreeBSD, you must buy a license for
> $ 100,-. I expect this money to be transferred to my bank account as
> soon as possible. The IBAN number is:
>
> 	NL30ABNA0385823426

You may get special hacker-support licence for $500/year, which allows you 
to get up to 25 responses to any questions from freebsd-hackers list, and 
any above 25 for just 15$.

From simon at optinet.com  Mon Jun 15 08:35:02 2009
From: simon at optinet.com (Simon)
Date: Mon Jun 15 08:35:09 2009
Subject: good morning to all
In-Reply-To: <alpine.BSF.2.00.0906150957250.17105@wojtek.tensor.gdynia.pl>
Message-ID: <20090615083501.BB1CB8FC0A@mx1.freebsd.org>

>> * malathi selvaraj <malathiramya@gmail.com> wrote:
>>> i am new one to  freeBSD, kindly guide me
>>
>> Sure. In order to be eligible to use FreeBSD, you must buy a license for
>> $ 100,-. I expect this money to be transferred to my bank account as
>> soon as possible. The IBAN number is:
>>
>> 	NL30ABNA0385823426

>You may get special hacker-support licence for $500/year, which allows you 
>to get up to 25 responses to any questions from freebsd-hackers list, and 
>any above 25 for just 15$.

Such a simple post, yet inspired so much creativity :-)

From wojtek at wojtek.tensor.gdynia.pl  Mon Jun 15 09:34:38 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun 15 09:34:44 2009
Subject: good morning to all
In-Reply-To: <20090615083501.BB1CB8FC0A@mx1.freebsd.org>
References: <20090615083501.BB1CB8FC0A@mx1.freebsd.org>
Message-ID: <alpine.BSF.2.00.0906151134030.17482@wojtek.tensor.gdynia.pl>

>
>> You may get special hacker-support licence for $500/year, which allows you
>> to get up to 25 responses to any questions from freebsd-hackers list, and
>> any above 25 for just 15$.
>
> Such a simple post, yet inspired so much creativity :-)

he should decide quickly, as it's limited offer of summer 2009 FreeBSD 
promotion ;)
From patfbsd at davenulle.org  Mon Jun 15 10:08:59 2009
From: patfbsd at davenulle.org (Patrick Lamaiziere)
Date: Mon Jun 15 10:09:06 2009
Subject: good morning to all
In-Reply-To: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>
References: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>
Message-ID: <20090615120858.06e1e941@baby-jane.lamaiziere.net>

Le Mon, 15 Jun 2009 11:22:17 +0530,
malathi selvaraj <malathiramya@gmail.com> a ?crit :

Hello,

> i am new one to  freeBSD, kindly guide me

See the Handbook, is good.

If you have questions, ask on the freebsd-questions@freebsd.org
mailing list (but you should read the doc before, it's better). There
are also some web forums http://forums.freebsd.org/

Regards.
From doconnor at gsoft.com.au  Mon Jun 15 11:59:50 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Mon Jun 15 11:59:59 2009
Subject: Booting from FAT(32)
In-Reply-To: <200906141410.14771.doconnor@gsoft.com.au>
References: <200906140015.44560.doconnor@gsoft.com.au>
	<200906141034.01890.doconnor@gsoft.com.au>
	<200906141410.14771.doconnor@gsoft.com.au>
Message-ID: <200906152129.44454.doconnor@gsoft.com.au>

WARNING: This e-mail has been altered by MIMEDefang.  Following this
paragraph are indications of the actual changes made.  For more
information about your site's MIMEDefang policy, contact
Postmaster <postmaster@gsoft.com.au>.  For more information about MIMEDefang, see:

            http://www.roaringpenguin.com/mimedefang/enduser.php3

An attachment named makeusb.sh was removed from this document as it
constituted a security hazard.  If you require this document, please contact
the sender and arrange an alternate means of receiving it.

-------------- next part --------------
Skipped content of type multipart/signed
From doconnor at gsoft.com.au  Mon Jun 15 13:28:23 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Mon Jun 15 13:28:30 2009
Subject: Booting from FAT(32)
In-Reply-To: <200906152129.44454.doconnor@gsoft.com.au>
References: <200906140015.44560.doconnor@gsoft.com.au>
	<200906141410.14771.doconnor@gsoft.com.au>
	<200906152129.44454.doconnor@gsoft.com.au>
Message-ID: <200906152258.16747.doconnor@gsoft.com.au>

On Mon, 15 Jun 2009, Daniel O'Connor wrote:
> WARNING: This e-mail has been altered by MIMEDefang.  Following this
> paragraph are indications of the actual changes made.  For more
> information about your site's MIMEDefang policy, contact
> Postmaster <postmaster@gsoft.com.au>.  For more information about
> MIMEDefang, see:
>
>             http://www.roaringpenguin.com/mimedefang/enduser.php3
>
> An attachment named makeusb.sh was removed from this document as it
> constituted a security hazard.  If you require this document, please
> contact the sender and arrange an alternate means of receiving it.

Ooops..
Try http://www.gsoft.com.au/~doconnor/makeusb.sh

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090615/7fdf2b09/attachment.pgp
From des at des.no  Mon Jun 15 15:15:26 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Mon Jun 15 15:15:34 2009
Subject: good morning to all
In-Reply-To: <20090615063638.GS48776@hoeg.nl> (Ed Schouten's message of "Mon, 
	15 Jun 2009 08:36:38 +0200")
References: <fc09aacb0906142252u4b6d206cj345a848d4018204c@mail.gmail.com>
	<20090615063638.GS48776@hoeg.nl>
Message-ID: <864ouhle9f.fsf@ds4.des.no>

Ed Schouten <ed@80386.nl> writes:
> Sure. In order to be eligible to use FreeBSD, you must buy a license for
> $ 100,-.

Reminds me of the guy who wrote to one of the lists once asking where he
could buy the full version of FreeBSD, since he only had the shareware
version that was limited to four VTYs.

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From pranavpeshwe at gmail.com  Mon Jun 15 15:42:21 2009
From: pranavpeshwe at gmail.com (Pranav Peshwe)
Date: Mon Jun 15 15:42:28 2009
Subject: good morning to all
In-Reply-To: <alpine.BSF.2.00.0906151134030.17482@wojtek.tensor.gdynia.pl>
References: <20090615083501.BB1CB8FC0A@mx1.freebsd.org>
	<alpine.BSF.2.00.0906151134030.17482@wojtek.tensor.gdynia.pl>
Message-ID: <cdfd7d6d0906150821o2af723cai23cc432a83e84aef@mail.gmail.com>

>From the name, I guess, its a 'she' not a 'he' :)


On 6/15/09, Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> wrote:
>>
>>> You may get special hacker-support licence for $500/year, which allows
>>> you
>>> to get up to 25 responses to any questions from freebsd-hackers list, and
>>> any above 25 for just 15$.
>>
>> Such a simple post, yet inspired so much creativity :-)
>
> he should decide quickly, as it's limited offer of summer 2009 FreeBSD
> promotion ;)
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>

-- 
Sent from my mobile device
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun 15 16:11:42 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun 15 16:11:49 2009
Subject: good morning to all
In-Reply-To: <cdfd7d6d0906150821o2af723cai23cc432a83e84aef@mail.gmail.com>
References: <20090615083501.BB1CB8FC0A@mx1.freebsd.org>
	<alpine.BSF.2.00.0906151134030.17482@wojtek.tensor.gdynia.pl>
	<cdfd7d6d0906150821o2af723cai23cc432a83e84aef@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906151811050.21048@wojtek.tensor.gdynia.pl>

thanks ;)

It's not that easy like in Polish - where you simply look if name ends 
with "a" for women.

On Mon, 15 Jun 2009, Pranav Peshwe wrote:

>> From the name, I guess, its a 'she' not a 'he' :)
>
>
> On 6/15/09, Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> wrote:
>>>
>>>> You may get special hacker-support licence for $500/year, which allows
>>>> you
>>>> to get up to 25 responses to any questions from freebsd-hackers list, and
>>>> any above 25 for just 15$.
>>>
>>> Such a simple post, yet inspired so much creativity :-)
>>
>> he should decide quickly, as it's limited offer of summer 2009 FreeBSD
>> promotion ;)
>> _______________________________________________
>> freebsd-hackers@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>>
>
> -- 
> Sent from my mobile device
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>
>
From delphij at delphij.net  Mon Jun 15 18:30:53 2009
From: delphij at delphij.net (Xin LI)
Date: Mon Jun 15 18:31:06 2009
Subject: tmpfs experimental?
In-Reply-To: <h162mo$jj2$1@ger.gmane.org>
References: <h162mo$jj2$1@ger.gmane.org>
Message-ID: <4A36930F.2000302@delphij.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ivan Voras wrote:
> Hi,
> 
> Are there still known problems with tmpfs?
> 
> I've been using it for a while in 7-STABLE and 8-CURRENT without
> noticeable problems - not that there was ever serious load involved
> (normal /tmp activity). I've just tried it and it survived a couple of
> rounds of blogbench, even with virtual memory swapping.
> 
> In other words, is there still reason for the "highly experimental
> feature" warning?

Last time when I added the warning, it was because some data corruption
issue that can be identified by fsx which I didn't got a chance to
investigate further.  I think tmpfs is Ok for some usual work but maybe
not ready for production at that moment.  alc@ and kib@ has made a lot
of changes on it recently so perhaps we need to re-visit the problems,
tmpfs would be a great feature for us.

Cheers,
- --
Xin LI <delphij@delphij.net>	http://www.delphij.net/
FreeBSD - The Power to Serve!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (FreeBSD)

iEYEARECAAYFAko2kw8ACgkQi+vbBBjt66D8LwCgiEevv8qy5pl/b73rDhXU6oso
jr0AoLKo/WGvoLOU7HrivC8KK2yidKo+
=alk6
-----END PGP SIGNATURE-----
From jh at saunalahti.fi  Mon Jun 15 19:10:03 2009
From: jh at saunalahti.fi (Jaakko Heinonen)
Date: Mon Jun 15 19:10:10 2009
Subject: tmpfs experimental?
In-Reply-To: <h162mo$jj2$1@ger.gmane.org>
References: <h162mo$jj2$1@ger.gmane.org>
Message-ID: <20090615185254.GA4303@a91-153-125-115.elisa-laajakaista.fi>

On 2009-06-15, Ivan Voras wrote:
> Are there still known problems with tmpfs?

I think sendfile(2) is still broken on tmpfs. See:

http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/127213

-- 
Jaakko
From mel.flynn+fbsd.hackers at mailing.thruhere.net  Mon Jun 15 21:53:11 2009
From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn)
Date: Mon Jun 15 21:53:17 2009
Subject: How best to debug locking/scheduler problems
Message-ID: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>

Hi,

I'm trying to get to the bottom of a bug with getpeername() and certain kde4 
applications which is probably as low-level as the libthr and the scheduler.

From browsing various related files in sys/kern it seems KTR is a good bet to 
get the information needed, yet it isn't really well supported in userland. 
For one, I've got no clue other then logging console output(?) how to retrieve 
the lock info or filter it in userland from reading ktr(9) and alq(9). Gdb is 
useless as the process doesn't give the information gdb wants and gdb just 
hangs in wait. ktrace also does not provide anything as there are no more 
syscalls being made, so I'll have to get to the bottom of this by tracing and 
filtering.

Short description of the problem:
a process never gets out of mi_switch and remains locked even init tries to 
shut it down.

% procstat -t 4283

  PID    TID COMM             TDNAME           CPU  PRI STATE   WCHAN    
 4283 100215 kdeinit4         -                  0  128 lock    *unp_mtx  
% procstat -k 4283

  PID    TID COMM             TDNAME           KSTACK                       
 4283 100215 kdeinit4         -                mi_switch turnstile_wait 
_mtx_lock_sleep uipc_peeraddr kern_getpeername getpeername syscall 
Xint0x80_syscall 
% ps -ww 4283
  PID  TT  STAT      TIME COMMAND
 4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http 
local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket local:/tmp/ksocket-
mel/plasmayC1653.slave-socket (kdeinit4)

%ls -l /tmp/ksocket-mel/

total 2
-rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55 klauncherxJ1635.slave-socket

-- 
Mel
From ben at wanderview.com  Tue Jun 16 03:45:21 2009
From: ben at wanderview.com (Ben Kelly)
Date: Tue Jun 16 04:32:01 2009
Subject: tmpfs experimental?
In-Reply-To: <h162mo$jj2$1@ger.gmane.org>
References: <h162mo$jj2$1@ger.gmane.org>
Message-ID: <4A370DB2.8070403@wanderview.com>

Ivan Voras wrote:
> Hi,
> 
> Are there still known problems with tmpfs?
> 
> I've been using it for a while in 7-STABLE and 8-CURRENT without
> noticeable problems - not that there was ever serious load involved
> (normal /tmp activity). I've just tried it and it survived a couple of
> rounds of blogbench, even with virtual memory swapping.
> 
> In other words, is there still reason for the "highly experimental
> feature" warning?

I get some slightly unexpected behavior when mount <mountpoint> is run
multiple times:

  ianto# mount | grep ' /tmp'
  tmpfs on /tmp (tmpfs, local)
  ianto# mount /tmp
  ianto# mount | grep ' /tmp'
  tmpfs on /tmp (tmpfs, local)
  tmpfs on /tmp (tmpfs, local)
  ianto# umount /tmp
  ianto# mount | grep ' /tmp'
  tmpfs on /tmp (tmpfs, local)
  ianto#

It also occurred to me once that perhaps all tmpfs mounts should share
the same UMA zones instead of a new zone for each mount, but thats a
pretty minor issue:

  ianto# vmstat -z | grep TMPFS
  TMPFS dirent:    20,        0,        4,      165,      385,        0
  TMPFS node:      136,        0,        5,       53,      386,        0
  TMPFS dirent:    20,        0,        4,      165,     5541,        0
  TMPFS node:      136,        0,        5,       53,     5542,        0
  TMPFS dirent:    20,        0,        6,      163,    51031,        0
  TMPFS node:      136,        0,        7,       80,    46927,        0
  TMPFS dirent:    20,        0,        4,      165,     7542,        0
  TMPFS node:      136,        0,        5,       53,     7543,        0
  TMPFS dirent:    20,        0,        6,      163,    81644,        0
  TMPFS node:      136,        0,        8,       79,    77463,        0


Overall tmpfs has been very stable for me as a mimedefang spool directory.

Hope that helps.

- Ben
From zbeeble at gmail.com  Tue Jun 16 05:50:11 2009
From: zbeeble at gmail.com (Zaphod Beeblebrox)
Date: Tue Jun 16 05:50:18 2009
Subject: Changes in the routing socket datagram between 7.0 and 7.2?
Message-ID: <5f67a8c40906152229t2123e8f1ma2c1ccafbb4f8e02@mail.gmail.com>

Did we change something in the routing socket's datagram between 7.0 and
7.2?  I have a binary I compiled on 7.0-RELEASE and it fails to add a route
on 7.2.  If I recompile the source on 7.2, it works.  Roughly put, the code
make a datagram for the route socket like this:

        bzero(&rtmsg, sizeof(rtmsg));

        /* Initial static part of the route message */
        rtmsg.mrtm.rtm_msglen = sizeof(rtmsg);
        rtmsg.mrtm.rtm_type = RTM_ADD;
        rtmsg.mrtm.rtm_flags = RTF_UP | RTF_STATIC | RTF_GATEWAY;
        rtmsg.mrtm.rtm_version = RTM_VERSION;
        rtmsg.mrtm.rtm_addrs = RTA_DST | RTA_GATEWAY | RTA_NETMASK;
        /* Gateway will always be the same */
        rtmsg.gateway.sin_family = AF_INET;
        rtmsg.gateway.sin_len = sizeof(rtmsg.gateway);

        /* Add a route to localhost for my address first */
        rtmsg.dest.sin_addr.s_addr = cons->LtunAddr;
        rtmsg.mask.sin_addr.s_addr = INADDR_BROADCAST;
        rtmsg.mrtm.rtm_seq = htons(fsd->routeSeq++);
        rtmsg.gateway.sin_addr.s_addr = INADDR_LOOPBACK;
From pranavpeshwe at gmail.com  Tue Jun 16 08:56:27 2009
From: pranavpeshwe at gmail.com (Pranav Peshwe)
Date: Tue Jun 16 08:56:34 2009
Subject: good morning to all
In-Reply-To: <alpine.BSF.2.00.0906151811050.21048@wojtek.tensor.gdynia.pl>
References: <20090615083501.BB1CB8FC0A@mx1.freebsd.org>
	<alpine.BSF.2.00.0906151134030.17482@wojtek.tensor.gdynia.pl>
	<cdfd7d6d0906150821o2af723cai23cc432a83e84aef@mail.gmail.com>
	<alpine.BSF.2.00.0906151811050.21048@wojtek.tensor.gdynia.pl>
Message-ID: <cdfd7d6d0906160156r3f61affdybdeafee343aeb5f3@mail.gmail.com>

On Mon, Jun 15, 2009 at 9:41 PM, Wojciech Puchar <
wojtek@wojtek.tensor.gdynia.pl> wrote:

> thanks ;)
>
> It's not that easy like in Polish - where you simply look if name ends with
> "a" for women.


There is somewhat similar a logic for Indian names too. Female Indian names
generally end in vowels. Males' generally don't.

- P
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun 16 11:30:57 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun 16 11:31:09 2009
Subject: tmpfs experimental?
In-Reply-To: <4A36930F.2000302@delphij.net>
References: <h162mo$jj2$1@ger.gmane.org> <4A36930F.2000302@delphij.net>
Message-ID: <alpine.BSF.2.00.0906161327350.27588@wojtek.tensor.gdynia.pl>

>> In other words, is there still reason for the "highly experimental
>> feature" warning?
>
> Last time when I added the warning, it was because some data corruption
> issue that can be identified by fsx which I didn't got a chance to
> investigate further.  I think tmpfs is Ok for some usual work but maybe
> not ready for production at that moment.  alc@ and kib@ has made a lot
> of changes on it recently so perhaps we need to re-visit the problems,
> tmpfs would be a great feature for us.

as an ordinary user not programmer of tmpfs i can say that:

1) runs fine for months in production environments, including case with 
over 40 mountpoints (jails)
2) runs really fast when memory is available.
3) performance is bad in case that swapping actually is used. It reads 
from swap with too small chunks. it's a place for improvement here.

Its great thing as it does it properly - memory is immediately freed on 
delete, and no caching of memory disk like with md(4).
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun 16 11:31:50 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun 16 11:32:02 2009
Subject: tmpfs experimental?
In-Reply-To: <4A370DB2.8070403@wanderview.com>
References: <h162mo$jj2$1@ger.gmane.org> <4A370DB2.8070403@wanderview.com>
Message-ID: <alpine.BSF.2.00.0906161330470.27588@wojtek.tensor.gdynia.pl>

> multiple times:
>
>  ianto# mount | grep ' /tmp'
>  tmpfs on /tmp (tmpfs, local)
>  ianto# mount /tmp
>  ianto# mount | grep ' /tmp'
>  tmpfs on /tmp (tmpfs, local)
>  tmpfs on /tmp (tmpfs, local)
>  ianto# umount /tmp
>  ianto# mount | grep ' /tmp'
>  tmpfs on /tmp (tmpfs, local)
>  ianto#

it's not only tmpfs. you may mount ufs readonly, or nfs multiple times on 
the same place.

It's not tmpfs specific problem IMHO. There is no checking in mount for 
that case.
From marius at nuenneri.ch  Tue Jun 16 13:20:36 2009
From: marius at nuenneri.ch (=?ISO-8859-1?Q?Marius_N=FCnnerich?=)
Date: Tue Jun 16 13:20:45 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <b649e5e0906160620i2ffb7200vb77f7f68fa292859@mail.gmail.com>

On Mon, Jun 15, 2009 at 23:53, Mel
Flynn<mel.flynn+fbsd.hackers@mailing.thruhere.net> wrote:
> Hi,
>
> I'm trying to get to the bottom of a bug with getpeername() and certain kde4
> applications which is probably as low-level as the libthr and the scheduler.
>
> From browsing various related files in sys/kern it seems KTR is a good bet to
> get the information needed, yet it isn't really well supported in userland.
> For one, I've got no clue other then logging console output(?) how to retrieve
> the lock info or filter it in userland from reading ktr(9) and alq(9). Gdb is
> useless as the process doesn't give the information gdb wants and gdb just
> hangs in wait. ktrace also does not provide anything as there are no more
> syscalls being made, so I'll have to get to the bottom of this by tracing and
> filtering.
>
> Short description of the problem:
> a process never gets out of mi_switch and remains locked even init tries to
> shut it down.
>

[snip]

Hi Mel,

my idea would be to try DTrace for this. Hopefully the following link
will help you:
http://wiki.freebsd.org/DTrace
There are more links at the bottom.

I added DTrace probes to geom a while ago and it's really easy. Feel
free to ask for more help.

Kind regards
Marius
From jhb at freebsd.org  Tue Jun 16 13:40:42 2009
From: jhb at freebsd.org (John Baldwin)
Date: Tue Jun 16 13:41:28 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <200906160819.57658.jhb@freebsd.org>

On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> Hi,
> 
> I'm trying to get to the bottom of a bug with getpeername() and certain kde4 
> applications which is probably as low-level as the libthr and the scheduler.
> 
> From browsing various related files in sys/kern it seems KTR is a good bet 
to 
> get the information needed, yet it isn't really well supported in userland. 
> For one, I've got no clue other then logging console output(?) how to 
retrieve 
> the lock info or filter it in userland from reading ktr(9) and alq(9). Gdb 
is 
> useless as the process doesn't give the information gdb wants and gdb just 
> hangs in wait. ktrace also does not provide anything as there are no more 
> syscalls being made, so I'll have to get to the bottom of this by tracing 
and 
> filtering.
> 
> Short description of the problem:
> a process never gets out of mi_switch and remains locked even init tries to 
> shut it down.
> 
> % procstat -t 4283
> 
>   PID    TID COMM             TDNAME           CPU  PRI STATE   WCHAN    
>  4283 100215 kdeinit4         -                  0  128 lock    *unp_mtx  
> % procstat -k 4283
> 
>   PID    TID COMM             TDNAME           KSTACK                       
>  4283 100215 kdeinit4         -                mi_switch turnstile_wait 
> _mtx_lock_sleep uipc_peeraddr kern_getpeername getpeername syscall 
> Xint0x80_syscall 
> % ps -ww 4283
>   PID  TT  STAT      TIME COMMAND
>  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http 
> local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket local:/tmp/ksocket-
> mel/plasmayC1653.slave-socket (kdeinit4)
> 
> %ls -l /tmp/ksocket-mel/
> 
> total 2
> -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55 klauncherxJ1635.slave-socket

You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb.  Simply 
run 'kgdb' as root and do 'lcd /folder/with/scripts' and 'source gdb6'.  You 
can then do 'lockchain 4283' to find who holds the lock this thread is 
blocked on and what state they are in.  You can use 'show lockchain 4283' in 
DDB for similar info as well.

-- 
John Baldwin
From doconnor at gsoft.com.au  Tue Jun 16 14:20:46 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Tue Jun 16 14:20:54 2009
Subject: Loader reading FAT
Message-ID: <200906162350.40221.doconnor@gsoft.com.au>

Hi,
I am attempting to get create a FAT FS USB stick to install FreeBSD 
from.

I have it working however I would like to pare back the size of the 
syslinux memdisk it loads. Currently it has the loader, kernel & 
sysinstall MFS which is not very small (this isn't an issue for me but 
I find it less elegant than I'd like).

I think it should be possible to just have the loader in the syslinux 
memdisk and then have that read the rest from the USB stick. 

Unfortunately I can't get the loader to read a FAT partition which 
surprises me because I think it should be able to.. I believe that 
libstand can do it (I can see the code :) however when I list the USB 
stick device I get an empty directory listing.

The partition is marked as type 0x0c (FAT32 LBA) and the loader sees it 
in lsdev. The FS was made with newfs_msdos with no arguments other than 
the device.

Any ideas?

Thanks.

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090616/e7def0db/attachment.pgp
From ben at wanderview.com  Tue Jun 16 14:10:44 2009
From: ben at wanderview.com (Ben Kelly)
Date: Tue Jun 16 15:17:48 2009
Subject: tmpfs experimental?
In-Reply-To: <h17pm9$4du$1@ger.gmane.org>
References: <h162mo$jj2$1@ger.gmane.org> <4A370DB2.8070403@wanderview.com>
	<h17pm9$4du$1@ger.gmane.org>
Message-ID: <4A37A7E1.6060905@wanderview.com>

Ivan Voras wrote:
> Ben Kelly wrote:
>> I get some slightly unexpected behavior when mount <mountpoint> is run
>> multiple times:
>>
>>   ianto# mount | grep ' /tmp'
>>   tmpfs on /tmp (tmpfs, local)
>>   ianto# mount /tmp
>>   ianto# mount | grep ' /tmp'
>>   tmpfs on /tmp (tmpfs, local)
>>   tmpfs on /tmp (tmpfs, local)
>>   ianto# umount /tmp
>>   ianto# mount | grep ' /tmp'
>>   tmpfs on /tmp (tmpfs, local)
> 
> Sorry, maybe I'm missing the point, but what is wrong with the above
> behaviour? AFAIK you are allowed to mount multiple file systems on the
> same directory, though it isn't very useful.

I guess I just found it confusing because it effectively made files
disappear from my file system.  This was a surprise to me since I was
interpreting fstab as saying "mount a single tmpfs file system here".
The only other file system I really had to compare it to was a writable
UFS partition which does not exhibit this behavior.

Anyway, I have no problem if this is expected behavior.  Just mentioning
what I thought was an oddity.  Overall its been very stable for me.

- Ben

From ajurna at gmail.com  Tue Jun 16 16:29:53 2009
From: ajurna at gmail.com (Peter Dwyer)
Date: Tue Jun 16 16:30:01 2009
Subject: Invitation from Peter Dwyer
Message-ID: <20090616162953.1E1358FC21@mx1.freebsd.org>

See my latest photos, updates and friends on Bebo. 

Click to view my profile. 

http://www.bebo.com/T/4.aaa24355-d6ae-4a84-8c23-b56d127954f2/inv/9464482779a941261774b9464320231c135de0

.......................................................................
This email was sent to you at the direct request of Peter Dwyer <ajurna@gmail.com>. You have not been added to a mailing list. 

If you would prefer not to receive invitations from ANY Bebo members please click here - http://www.bebo.com/T/4.aaa24355-d6ae-4a84-8c23-b56d127954f2/unsub/9464482779a941261774 

Bebo, Inc., 795 Folsom St, 6th Floor, San Francisco, CA 94107, USA.
From rick-freebsd2008 at kiwi-computer.com  Tue Jun 16 17:49:49 2009
From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty)
Date: Tue Jun 16 17:49:56 2009
Subject: Loader reading FAT
In-Reply-To: <200906162350.40221.doconnor@gsoft.com.au>
References: <200906162350.40221.doconnor@gsoft.com.au>
Message-ID: <20090616172306.GA91395@keira.kiwi-computer.com>

On Tue, Jun 16, 2009 at 11:50:38PM +0930, Daniel O'Connor wrote:
> 
> Unfortunately I can't get the loader to read a FAT partition which 
> surprises me because I think it should be able to.. I believe that 
> libstand can do it (I can see the code :) however when I list the USB 
> stick device I get an empty directory listing.

I read somewhere that there isn't enough space in the boot2 loader to put
such logic.  You're only guaranteed 15 512-byte sectors or 7680 bytes, if
you use any UFS partition.  It's pretty tight; I think you will find it
difficult to insert another file system in there, especially one as
complicated as msdos.  libstand is 223 KB, so it's not as trivial as you
think.

Theoretically it would be possible: for example, if you're willing to set
aside a separate partition you would have as much room as you want.  Or if
you put it at the front of a UFS partition, you have just under 256 KB of
room since our UFS code will search for the superblock at a byte offset of
262144, but there aren't any knobs to newfs so you'd have to hack it
together.  Take a look at /usr/src/sys/boot/i386/boot2/ for starters.

-- Rick C. Petty
From mel.flynn+fbsd.hackers at mailing.thruhere.net  Tue Jun 16 17:58:00 2009
From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn)
Date: Tue Jun 16 17:58:08 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906160819.57658.jhb@freebsd.org>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906160819.57658.jhb@freebsd.org>
Message-ID: <200906160952.24895.mel.flynn+fbsd.hackers@mailing.thruhere.net>

Hi John,

On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:

> >   PID    TID COMM             TDNAME           KSTACK
> >  4283 100215 kdeinit4         -                mi_switch turnstile_wait
> > _mtx_lock_sleep uipc_peeraddr kern_getpeername getpeername syscall
> > Xint0x80_syscall
> > % ps -ww 4283
> >   PID  TT  STAT      TIME COMMAND
> >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket local:/tmp/ksocket-
> > mel/plasmayC1653.slave-socket (kdeinit4)
> >
> > %ls -l /tmp/ksocket-mel/
> >
> > total 2
> > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55 klauncherxJ1635.slave-socket
>
> You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb.  Simply
> run 'kgdb' as root and do 'lcd /folder/with/scripts' and 'source gdb6'. 
> You can then do 'lockchain 4283' to find who holds the lock this thread is
> blocked on and what state they are in.

Looks like a deadlock:

(kgdb) lockchain 4283
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
 thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
 DEADLOCK

Looking through the scripts now to see how I can get more info on the call 
chain and hoping I don't panic the machine ;). It is quite random to 
reproduce.
-- 
Mel
From jhb at freebsd.org  Tue Jun 16 19:07:40 2009
From: jhb at freebsd.org (John Baldwin)
Date: Tue Jun 16 19:07:47 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906160952.24895.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906160819.57658.jhb@freebsd.org>
	<200906160952.24895.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <200906161502.42741.jhb@freebsd.org>

On Tuesday 16 June 2009 1:52:23 pm Mel Flynn wrote:
> Hi John,
> 
> On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> 
> > >   PID    TID COMM             TDNAME           KSTACK
> > >  4283 100215 kdeinit4         -                mi_switch turnstile_wait
> > > _mtx_lock_sleep uipc_peeraddr kern_getpeername getpeername syscall
> > > Xint0x80_syscall
> > > % ps -ww 4283
> > >   PID  TT  STAT      TIME COMMAND
> > >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket local:/tmp/ksocket-
> > > mel/plasmayC1653.slave-socket (kdeinit4)
> > >
> > > %ls -l /tmp/ksocket-mel/
> > >
> > > total 2
> > > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55 klauncherxJ1635.slave-socket
> >
> > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb.  Simply
> > run 'kgdb' as root and do 'lcd /folder/with/scripts' and 'source gdb6'. 
> > You can then do 'lockchain 4283' to find who holds the lock this thread is
> > blocked on and what state they are in.
> 
> Looks like a deadlock:
> 
> (kgdb) lockchain 4283
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
>  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
>  DEADLOCK
> 
> Looking through the scripts now to see how I can get more info on the call 
> chain and hoping I don't panic the machine ;). It is quite random to 
> reproduce.

In kgdb you can simply do 'tid 100122' followed by 'bt' and 'tid 100215' 
followed by 'bt'.

-- 
John Baldwin
From mel.flynn+fbsd.hackers at mailing.thruhere.net  Tue Jun 16 23:01:48 2009
From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn)
Date: Tue Jun 16 23:01:55 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906161502.42741.jhb@freebsd.org>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906160952.24895.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906161502.42741.jhb@freebsd.org>
Message-ID: <200906161501.45411.mel.flynn+fbsd.hackers@mailing.thruhere.net>

On Tuesday 16 June 2009 11:02:42 John Baldwin wrote:
> On Tuesday 16 June 2009 1:52:23 pm Mel Flynn wrote:
> > Hi John,
> >
> > On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> > > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> > > >   PID    TID COMM             TDNAME           KSTACK
> > > >  4283 100215 kdeinit4         -                mi_switch
> > > > turnstile_wait _mtx_lock_sleep uipc_peeraddr kern_getpeername
> > > > getpeername syscall Xint0x80_syscall
> > > > % ps -ww 4283
> > > >   PID  TT  STAT      TIME COMMAND
> > > >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket
> > > > local:/tmp/ksocket- mel/plasmayC1653.slave-socket (kdeinit4)
> > > >
> > > > %ls -l /tmp/ksocket-mel/
> > > >
> > > > total 2
> > > > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > > > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > > > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55
> > > > klauncherxJ1635.slave-socket
> > >
> > > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb.  Simply
> > > run 'kgdb' as root and do 'lcd /folder/with/scripts' and 'source gdb6'.
> > > You can then do 'lockchain 4283' to find who holds the lock this thread
> > > is blocked on and what state they are in.
> >
> > Looks like a deadlock:
> >
> > (kgdb) lockchain 4283
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> >  DEADLOCK
> >
> > Looking through the scripts now to see how I can get more info on the
> > call chain and hoping I don't panic the machine ;). It is quite random to
> > reproduce.
>
> In kgdb you can simply do 'tid 100122' followed by 'bt' and 'tid 100215'
> followed by 'bt'.

Cool, thanks for helping John. Of course it pretty much shows me what procstat 
-k shows and can't get any info on the userland part, but I can fully inspect 
the locks and threads.

Both threads are in TDS_INHIBITED state, and blocked on:
(kgdb) frame 0
#0  sched_switch (td=0xc5971240, newtd=0xc4d39900, flags=259)
    at /usr/src/sys/kern/sched_ule.c:1864
1864                    cpuid = PCPU_GET(cpuid);

print newtd->td_name
$9 = "idle: cpu0\000\000\000\000\000\000\000\000\000"

Is there anything you want to see to shed some light on why these threads 
might be deadlocked?

This is a 8-current kernel, seen this issue for a while (<March for sure) but 
the running one is FreeBSD 8.0-CURRENT #2 r194183: Sun Jun 14 15:09:27 AKDT 
2009. Not a GENERIC, basically stripped I[45]86_CPU, SCTP, unused hardware, no 
PRINTF_BUFFER_SIZE, added wpi, ichsmb and smbus, mmc, mmcsd and sdhci, HWPMC. 
Config inlined below sig.


-- 
Mel

#
# GENERIC -- Generic kernel configuration file for FreeBSD/i386
#
# For more information on this file, please read the config(5) manual page,
# and/or the handbook section on Kernel Configuration Files:
#
#    http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-
config.html
#
# The handbook is also available locally in /usr/share/doc/handbook
# if you've installed the doc distribution, otherwise always see the
# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the
# latest information.
#
# An exhaustive list of options and more detailed explanations of the
# device lines is also present in the ../../conf/NOTES and NOTES files.
# If you are in doubt as to the purpose or necessity of a line, check first
# in NOTES.
#
# Used:
# $FreeBSD: src/sys/i386/conf/GENERIC,v 1.511 2009/03/19 20:33:26 thompsa Exp 
$
# $FreeBSD: src/sys/conf/NOTES,v 1.1534 2009/04/15 22:38:22 marcel Exp $
#
# This file:
# $Coar: kernels/8.x/i386/SMOOCHIES,v 1.1 2009/04/17 11:50:10 mel Exp $

cpu		I686_CPU
ident		SMOOCHIES

# To statically compile in device wiring instead of /boot/device.hints
#hints		"GENERIC.hints"		# Default places to look for devices.

# Use the following to compile in values accessible to the kernel
# through getenv() (or kenv(1) in userland). The format of the file
# is 'variable=value', see kenv(1)
#
# env		"GENERIC.env"

makeoptions	DEBUG=-g		# Build kernel with gdb(1) debug symbols

#
# MAXPHYS and DFLTPHYS
#
# These are the max and default 'raw' I/O block device access sizes.
# Reads and writes will be split into DFLTPHYS chunks. Some applications
# have better performance with larger raw I/O access sizes. Typically
# MAXPHYS should be twice the size of DFLTPHYS. Note that certain VM
# parameters are derived from these values and making them too large
# can make an an unbootable kernel.
#
# The defaults are 64K and 128K respectively.
#options 	DFLTPHYS=(128*1024)
#options 	MAXPHYS=(256*1024)

options 	INCLUDE_CONFIG_FILE     # Include this file in kernel
options 	SCHED_ULE		# ULE scheduler
options 	PREEMPTION		# Enable kernel thread preemption
options 	INET			# InterNETworking
options 	INET6			# IPv6 communications protocols
options 	FFS			# Berkeley Fast Filesystem
options 	SOFTUPDATES		# Enable FFS soft updates support
options 	UFS_ACL			# Support for access control lists
options 	UFS_DIRHASH		# Improve performance on big directories
options 	UFS_GJOURNAL		# Enable gjournal-based UFS journaling
options 	MD_ROOT			# MD is a potential root device
options 	NFSCLIENT		# Network Filesystem Client
options 	NFSSERVER		# Network Filesystem Server
options 	NFSLOCKD		# Network Lock Manager
options 	NFS_ROOT		# NFS usable as /, requires NFSCLIENT
options 	MSDOSFS			# MSDOS Filesystem
options 	CD9660			# ISO 9660 Filesystem
options 	PROCFS			# Process filesystem (requires PSEUDOFS)
options 	PSEUDOFS		# Pseudo-filesystem framework
options 	GEOM_PART_GPT		# GUID Partition Tables.
options 	GEOM_LABEL		# Provides labelization
options 	COMPAT_43TTY		# BSD 4.3 TTY compat (sgtty)
options 	COMPAT_FREEBSD4		# Compatible with FreeBSD4
options 	COMPAT_FREEBSD5		# Compatible with FreeBSD5
options 	COMPAT_FREEBSD6		# Compatible with FreeBSD6
options 	COMPAT_FREEBSD7		# Compatible with FreeBSD7
options 	SCSI_DELAY=5000		# Delay (in ms) before probing SCSI
options 	KTRACE			# ktrace(1) support
options 	STACK			# stack(9) support
options 	SYSVSHM			# SYSV-style shared memory
options 	SYSVMSG			# SYSV-style message queues
options 	SYSVSEM			# SYSV-style semaphores
options 	_KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
options 	STOP_NMI		# Stop CPUS using NMI instead of IPI
options  	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
options 	AUDIT			# Security event auditing
#options 	KDTRACE_HOOKS		# Kernel DTrace hooks

# To make an SMP kernel, the next two lines are needed
options 	SMP			# Symmetric MultiProcessor Kernel
device		apic			# I/O APIC

# CPU frequency control
device		cpufreq

# Bus support.
device		acpi
device		eisa
device		pci

# ATA and ATAPI devices
device		ata
device		atadisk		# ATA disk drives
device		ataraid		# ATA RAID drives
device		atapicd		# ATAPI CDROM drives
device		atapifd		# ATAPI floppy drives
device		atapist		# ATAPI tape drives
options 	ATA_STATIC_ID	# Static device numbering

# SCSI peripherals
device		scbus		# SCSI bus (required for SCSI)
device		ch		# SCSI media changers
device		da		# Direct Access (disks)
device		sa		# Sequential Access (tape etc)
device		cd		# CD
device		pass		# Passthrough device (direct SCSI access)
device		ses		# SCSI Environmental Services (and SAF-TE)

# atkbdc0 controls both the keyboard and the PS/2 mouse
device		atkbdc		# AT keyboard controller
device		atkbd		# AT keyboard
device		psm		# PS/2 mouse

device		kbdmux		# keyboard multiplexer

device		vga		# VGA video card driver

device		splash		# Splash screen and screen saver support

# syscons is the default console driver, resembling an SCO console
device		sc

device		agp		# support several AGP chipsets

# Power management support (see NOTES for more options)
#device		apm
# Add suspend/resume support for the i8254.
device		pmtimer

# PCCARD (PCMCIA) support
# PCMCIA and cardbus bridge support
device		cbb		# cardbus (yenta) bridge
device		pccard		# PC Card (16-bit) bus
device		cardbus		# CardBus (32-bit) bus

# Serial (COM) ports
device		uart		# Generic UART driver

# If you've got a "dumb" serial or parallel PCI card that is
# supported by the puc(4) glue driver, uncomment the following
# line to enable it (connects to sio, uart and/or ppc drivers):
#device		puc

# PCI Ethernet NICs.
device		em		# Intel PRO/1000 Gigabit Ethernet Family
device		igb		# Intel PRO/1000 PCIE Server Gigabit Family
device		ixgb		# Intel PRO/10GbE Ethernet Card

# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device		miibus		# MII bus support

# Wireless NIC cards
device		wlan		# 802.11 support
options 	IEEE80211_DEBUG	# enable debug msgs
options 	IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's
device		wlan_wep	# 802.11 WEP support
device		wlan_ccmp	# 802.11 CCMP support
device		wlan_tkip	# 802.11 TKIP support
device		wlan_amrr	# AMRR transmit rate control algorithm
device		wpi		# Intel 3945ABG

# Pseudo devices.
device		loop		# Network loopback
device		random		# Entropy device
device		ether		# Ethernet support
device		tun		# Packet tunnel.
device		pty		# BSD-style compatibility pseudo ttys
device		md		# Memory "disks"
device		gif		# IPv6 and IPv4 tunneling
device		faith		# IPv6-to-IPv4 relaying (translation)
device		firmware	# firmware assist module

# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device		bpf		# Berkeley packet filter

# USB support
device		uhci		# UHCI PCI->USB interface
device		ohci		# OHCI PCI->USB interface
device		ehci		# EHCI PCI->USB interface (USB 2.0)
device		usb		# USB Bus (required)
#device		udbp		# USB Double Bulk Pipe devices
device		uhid		# "Human Interface Devices"
device		ukbd		# Keyboard
device		ulpt		# Printer
device		umass		# Disks/Mass storage - Requires scbus and da
device		ums		# Mouse
device		urio		# Diamond Rio 500 MP3 player
# USB Serial devices
device		u3g		# USB-based 3G modems (Option, Huawei, Sierra)
device		uark		# Technologies ARK3116 based serial adapters
device		ubsa		# Belkin F5U103 and compatible serial adapters
device		uftdi		# For FTDI usb serial adapters
device		uipaq		# Some WinCE based devices
device		uplcom		# Prolific PL-2303 serial adapters
device		uslcom		# SI Labs CP2101/CP2102 serial adapters
device		uvisor		# Visor and Palm devices
device		uvscom		# USB serial support for DDI pocket's PHS
# USB Ethernet, requires miibus
device		aue		# ADMtek USB Ethernet
device		axe		# ASIX Electronics USB Ethernet
device		cdce		# Generic USB over Ethernet
device		cue		# CATC USB Ethernet
device		kue		# Kawasaki LSI USB Ethernet
device		rue		# RealTek RTL8150 USB Ethernet
device		udav		# Davicom DM9601E USB

# FireWire support
device		firewire	# FireWire bus code
device		sbp		# SCSI over FireWire (Requires scbus and da)
device		fwe		# Ethernet over FireWire (non-standard!)
device		fwip		# IP over FireWire (RFC 2734,3146)
device		dcons		# Dumb console driver
device		dcons_crom	# Configuration ROM for dcons

#
# SMB bus
#
# System Management Bus support is provided by the 'smbus' device.
# Access to the SMBus device is via the 'smb' device (/dev/smb*),
# which is a child of the 'smbus' device.
#
# Supported devices:
# smb		standard I/O through /dev/smb*
#
# Supported SMB interfaces:
# iicsmb	I2C to SMB bridge with any iicbus interface
# bktr		brooktree848 I2C hardware interface
# intpm		Intel PIIX4 (82371AB, 82443MX) Power Management Unit
# alpm		Acer Aladdin-IV/V/Pro2 Power Management Unit
# ichsmb	Intel ICH SMBus controller chips (82801AA, 82801AB, 82801BA)
# viapm		VIA VT82C586B/596B/686A and VT8233 Power Management Unit
# amdpm		AMD 756 Power Management Unit
# amdsmb	AMD 8111 SMBus 2.0 Controller
# nfpm		NVIDIA nForce Power Management Unit
# nfsmb		NVIDIA nForce2/3/4 MCP SMBus 2.0 Controller
#
device		smbus		# Bus support, required for smb below.
device		ichsmb

#
# MMC/SD
#
# mmc 		MMC/SD bus
# mmcsd		MMC/SD memory card
# sdhci		Generic PCI SD Host Controller
#
device		mmc
device		mmcsd
device		sdhci

From doconnor at gsoft.com.au  Wed Jun 17 02:19:20 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Wed Jun 17 02:19:27 2009
Subject: Loader reading FAT
In-Reply-To: <20090616172306.GA91395@keira.kiwi-computer.com>
References: <200906162350.40221.doconnor@gsoft.com.au>
	<20090616172306.GA91395@keira.kiwi-computer.com>
Message-ID: <200906171149.13433.doconnor@gsoft.com.au>

On Wed, 17 Jun 2009, Rick C. Petty wrote:
> On Tue, Jun 16, 2009 at 11:50:38PM +0930, Daniel O'Connor wrote:
> > Unfortunately I can't get the loader to read a FAT partition which
> > surprises me because I think it should be able to.. I believe that
> > libstand can do it (I can see the code :) however when I list the
> > USB stick device I get an empty directory listing.
>
> I read somewhere that there isn't enough space in the boot2 loader to
> put such logic.  You're only guaranteed 15 512-byte sectors or 7680
> bytes, if you use any UFS partition.  It's pretty tight; I think you
> will find it difficult to insert another file system in there,
> especially one as complicated as msdos.  libstand is 223 KB, so it's
> not as trivial as you think.
>
> Theoretically it would be possible: for example, if you're willing to
> set aside a separate partition you would have as much room as you
> want.  Or if you put it at the front of a UFS partition, you have
> just under 256 KB of room since our UFS code will search for the
> superblock at a byte offset of 262144, but there aren't any knobs to
> newfs so you'd have to hack it together.  Take a look at
> /usr/src/sys/boot/i386/boot2/ for starters.

In effect I will be doing that, I can make a syslinux memdisk which 
holds boot0/1/2 + loader in a UFS partition. I would just prefer to be 
able to read the kernel, modules & MFS root off FAT32.

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090617/4a7afa3d/attachment.pgp
From adrian at freebsd.org  Wed Jun 17 12:09:13 2009
From: adrian at freebsd.org (Adrian Chadd)
Date: Wed Jun 17 12:09:20 2009
Subject: Loader reading FAT
In-Reply-To: <20090616172306.GA91395@keira.kiwi-computer.com>
References: <200906162350.40221.doconnor@gsoft.com.au>
	<20090616172306.GA91395@keira.kiwi-computer.com>
Message-ID: <d763ac660906170509rb2ed252i7bec83decb971e69@mail.gmail.com>

2009/6/17 Rick C. Petty <rick-freebsd2008@kiwi-computer.com>:

> I read somewhere that there isn't enough space in the boot2 loader to put
> such logic. ?You're only guaranteed 15 512-byte sectors or 7680 bytes, if
> you use any UFS partition. ?It's pretty tight; I think you will find it
> difficult to insert another file system in there, especially one as
> complicated as msdos. ?libstand is 223 KB, so it's not as trivial as you
> think.
>
> Theoretically it would be possible: for example, if you're willing to set
> aside a separate partition you would have as much room as you want. ?Or if
> you put it at the front of a UFS partition, you have just under 256 KB of
> room since our UFS code will search for the superblock at a byte offset of
> 262144, but there aren't any knobs to newfs so you'd have to hack it
> together. ?Take a look at /usr/src/sys/boot/i386/boot2/ for starters.

This is why I pointed out the NetBSD DOS bootloader stuff. In effect,
you could "just" boot a FreeBSD install by teaching the NetBSD
bootloader and loader(8) enough about each other to initialise loader;
loader then has much more memory to play with understanding multiple
filesystems and slicing/labelling methods. Boot0, boot1, boot2
wouldn't even be involved.


Adrian
From jhb at freebsd.org  Wed Jun 17 12:29:42 2009
From: jhb at freebsd.org (John Baldwin)
Date: Wed Jun 17 12:29:58 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906161501.45411.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906161502.42741.jhb@freebsd.org>
	<200906161501.45411.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <200906170815.26650.jhb@freebsd.org>

On Tuesday 16 June 2009 7:01:45 pm Mel Flynn wrote:
> On Tuesday 16 June 2009 11:02:42 John Baldwin wrote:
> > On Tuesday 16 June 2009 1:52:23 pm Mel Flynn wrote:
> > > Hi John,
> > >
> > > On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> > > > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> > > > >   PID    TID COMM             TDNAME           KSTACK
> > > > >  4283 100215 kdeinit4         -                mi_switch
> > > > > turnstile_wait _mtx_lock_sleep uipc_peeraddr kern_getpeername
> > > > > getpeername syscall Xint0x80_syscall
> > > > > % ps -ww 4283
> > > > >   PID  TT  STAT      TIME COMMAND
> > > > >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > > > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket
> > > > > local:/tmp/ksocket- mel/plasmayC1653.slave-socket (kdeinit4)
> > > > >
> > > > > %ls -l /tmp/ksocket-mel/
> > > > >
> > > > > total 2
> > > > > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > > > > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > > > > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55
> > > > > klauncherxJ1635.slave-socket
> > > >
> > > > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb.  Simply
> > > > run 'kgdb' as root and do 'lcd /folder/with/scripts' and 'source gdb6'.
> > > > You can then do 'lockchain 4283' to find who holds the lock this thread
> > > > is blocked on and what state they are in.
> > >
> > > Looks like a deadlock:
> > >
> > > (kgdb) lockchain 4283
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > >  thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > >  DEADLOCK
> > >
> > > Looking through the scripts now to see how I can get more info on the
> > > call chain and hoping I don't panic the machine ;). It is quite random to
> > > reproduce.
> >
> > In kgdb you can simply do 'tid 100122' followed by 'bt' and 'tid 100215'
> > followed by 'bt'.
> 
> Cool, thanks for helping John. Of course it pretty much shows me what procstat 
> -k shows and can't get any info on the userland part, but I can fully inspect 
> the locks and threads.
> 
> Both threads are in TDS_INHIBITED state, and blocked on:
> (kgdb) frame 0
> #0  sched_switch (td=0xc5971240, newtd=0xc4d39900, flags=259)
>     at /usr/src/sys/kern/sched_ule.c:1864
> 1864                    cpuid = PCPU_GET(cpuid);

That doesn't really tell us anything except that it isn't running.  We know
it is actually blocked on a lock, and we need the full stack trace to see
where the two threads were trying to acquire the locks, hence 'bt'.  '
procstat -k' output would be fine, too.

-- 
John Baldwin
From marck at rinet.ru  Wed Jun 17 15:46:25 2009
From: marck at rinet.ru (Dmitry Morozovsky)
Date: Wed Jun 17 15:46:38 2009
Subject: tmpfs experimental?
In-Reply-To: <alpine.BSF.2.00.0906161327350.27588@wojtek.tensor.gdynia.pl>
References: <h162mo$jj2$1@ger.gmane.org> <4A36930F.2000302@delphij.net>
	<alpine.BSF.2.00.0906161327350.27588@wojtek.tensor.gdynia.pl>
Message-ID: <alpine.BSF.2.00.0906171942200.6042@woozle.rinet.ru>

On Tue, 16 Jun 2009, Wojciech Puchar wrote:

WP> > > In other words, is there still reason for the "highly experimental
WP> > > feature" warning?
WP> > 
WP> > Last time when I added the warning, it was because some data corruption
WP> > issue that can be identified by fsx which I didn't got a chance to
WP> > investigate further.  I think tmpfs is Ok for some usual work but maybe
WP> > not ready for production at that moment.  alc@ and kib@ has made a lot
WP> > of changes on it recently so perhaps we need to re-visit the problems,
WP> > tmpfs would be a great feature for us.
WP> 
WP> as an ordinary user not programmer of tmpfs i can say that:
WP> 
WP> 1) runs fine for months in production environments, including case with over
WP> 40 mountpoints (jails)
WP> 2) runs really fast when memory is available.
WP> 3) performance is bad in case that swapping actually is used. It reads from
WP> swap with too small chunks. it's a place for improvement here.
WP> 
WP> Its great thing as it does it properly - memory is immediately freed on
WP> delete, and no caching of memory disk like with md(4).

Actually, buffer cache is used, so excessive memory usage are still in place; 
also, on rather heavy tmpfs usage (building large ports, for example) I still 
can panic and/or hang the machione with exhausted maxswzone, so there 
definitely is a place to improve things ;)

-- 
Sincerely,
D.Marck                                     [DM5020, MCK-RIPE, DM3-RIPN]
[ FreeBSD committer:                                 marck@FreeBSD.org ]
------------------------------------------------------------------------
*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru ***
------------------------------------------------------------------------
From alexbestms at math.uni-muenster.de  Wed Jun 17 19:24:56 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Wed Jun 17 19:25:03 2009
Subject: small usr.bin/find patch
Message-ID: <permail-200906171855491e86ffa800003434-a_best01@message-id.uni-muenster.de>

hi everybody,

here's just a quick hack i applied to find. i very often use the -size switch
and always forget if you need to append "m" or "M" for megabyte or "k" or "K"
for kilobyte. after applying the patch find accepts both. ;-)

cheers.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: findpatch
Type: application/octet-stream
Size: 941 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090617/0ebc90cb/findpatch.obj
From mel.flynn+fbsd.hackers at mailing.thruhere.net  Wed Jun 17 19:52:57 2009
From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn)
Date: Wed Jun 17 19:53:05 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906170815.26650.jhb@freebsd.org>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906161501.45411.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906170815.26650.jhb@freebsd.org>
Message-ID: <200906171152.55438.mel.flynn+fbsd.hackers@mailing.thruhere.net>

On Wednesday 17 June 2009 04:15:26 John Baldwin wrote:
> On Tuesday 16 June 2009 7:01:45 pm Mel Flynn wrote:
> > On Tuesday 16 June 2009 11:02:42 John Baldwin wrote:
> > > On Tuesday 16 June 2009 1:52:23 pm Mel Flynn wrote:
> > > > Hi John,
> > > >
> > > > On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> > > > > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> > > > > >   PID    TID COMM             TDNAME           KSTACK
> > > > > >  4283 100215 kdeinit4         -                mi_switch
> > > > > > turnstile_wait _mtx_lock_sleep uipc_peeraddr kern_getpeername
> > > > > > getpeername syscall Xint0x80_syscall
> > > > > > % ps -ww 4283
> > > > > >   PID  TT  STAT      TIME COMMAND
> > > > > >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > > > > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket
> > > > > > local:/tmp/ksocket- mel/plasmayC1653.slave-socket (kdeinit4)
> > > > > >
> > > > > > %ls -l /tmp/ksocket-mel/
> > > > > >
> > > > > > total 2
> > > > > > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > > > > > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > > > > > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55
> > > > > > klauncherxJ1635.slave-socket
> > > > >
> > > > > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb. 
> > > > > Simply run 'kgdb' as root and do 'lcd /folder/with/scripts' and
> > > > > 'source gdb6'. You can then do 'lockchain 4283' to find who holds
> > > > > the lock this thread is blocked on and what state they are in.
> > > >
> > > > Looks like a deadlock:
> > > >
> > > > (kgdb) lockchain 4283
> > > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on
> > > > lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked
> > > > on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4)
> > > > blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635,
> > > > klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid
> > > > 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122
> > > > (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread
> > > > 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > > > thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348
> > > > "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock
> > > > 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on
> > > > lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked
> > > > on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher)
> > > > blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283,
> > > > kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid
> > > > 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215
> > > > (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread
> > > > 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > > > thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > 0xc6806348 "unp_mtx" DEADLOCK
> > > >
> > > > Looking through the scripts now to see how I can get more info on the
> > > > call chain and hoping I don't panic the machine ;). It is quite
> > > > random to reproduce.
> > >
> > > In kgdb you can simply do 'tid 100122' followed by 'bt' and 'tid
> > > 100215' followed by 'bt'.
> >
> > Cool, thanks for helping John. Of course it pretty much shows me what
> > procstat -k shows and can't get any info on the userland part, but I can
> > fully inspect the locks and threads.
> >
> > Both threads are in TDS_INHIBITED state, and blocked on:
> > (kgdb) frame 0
> > #0  sched_switch (td=0xc5971240, newtd=0xc4d39900, flags=259)
> >     at /usr/src/sys/kern/sched_ule.c:1864
> > 1864                    cpuid = PCPU_GET(cpuid);
>
> That doesn't really tell us anything except that it isn't running.  We know
> it is actually blocked on a lock, and we need the full stack trace to see
> where the two threads were trying to acquire the locks, hence 'bt'.  '
> procstat -k' output would be fine, too.

They're blocking on each other:
(kgdb) tid 100122
(kgdb) print td->td_contested.lh_first
$16 = (struct turnstile *) 0xc4f8fb00
(kgdb) print td->td_lock
$17 = (struct mtx * volatile) 0xc538cb00
(kgdb) tid 100215
(kgdb) print td->td_contested.lh_first
$18 = (struct turnstile *) 0xc538cb00
(kgdb) print td->td_lock
$19 = (struct mtx * volatile) 0xc4f8fb00

the respective bt's:
(kgdb) tid 100122
#0  sched_switch (td=0xc56e8900, newtd=0xc4d39b40, flags=259)
    at /usr/src/sys/kern/sched_ule.c:1864
#1  0xc064bcfa in mi_switch (flags=259, newtd=0x0) at 
/usr/src/sys/kern/kern_synch.c:444
#2  0xc067d30b in turnstile_wait (ts=0xc538cb00, owner=0xc5971240, 
queue=Variable "queue" is not available.
)
    at /usr/src/sys/kern/subr_turnstile.c:745
#3  0xc06346ac in _mtx_lock_sleep (m=0xc6806348, tid=3312355584, opts=0, 
file=0x0, line=0)
    at /usr/src/sys/kern/kern_mutex.c:447
#4  0xc06a68a5 in uipc_peeraddr (so=0xc64309a8, nam=0xe79a2c70)
    at /usr/src/sys/kern/uipc_usrreq.c:682
#5  0xc06a1e71 in kern_getpeername (td=0xc56e8900, fd=12, sa=0xe79a2c70, 
alen=0xe79a2c6c)
    at /usr/src/sys/kern/uipc_syscalls.c:1566
#6  0xc06a1f72 in getpeername (td=0xc56e8900, uap=0xe79a2cf8)
    at /usr/src/sys/kern/uipc_syscalls.c:1527
#7  0xc0872415 in syscall (frame=0xe79a2d38) at 
/usr/src/sys/i386/i386/trap.c:1073
#8  0xc0856970 in Xint0x80_syscall () at 
/usr/src/sys/i386/i386/exception.s:261
#9  0x00000033 in ?? ()

(kgdb) tid 100215
(kgdb) bt
#0  sched_switch (td=0xc5971240, newtd=0xc4d39900, flags=259)
    at /usr/src/sys/kern/sched_ule.c:1864
#1  0xc064bcfa in mi_switch (flags=259, newtd=0x0) at 
/usr/src/sys/kern/kern_synch.c:444
#2  0xc067d30b in turnstile_wait (ts=0xc4f8fb00, owner=0xc56e8900, 
queue=Variable "queue" is not available.
)
    at /usr/src/sys/kern/subr_turnstile.c:745
#3  0xc06346ac in _mtx_lock_sleep (m=0xc64374a0, tid=3315012160, opts=0, 
file=0x0, line=0)
    at /usr/src/sys/kern/kern_mutex.c:447
#4  0xc06a68a5 in uipc_peeraddr (so=0xc6976338, nam=0xe9ae9c70)
    at /usr/src/sys/kern/uipc_usrreq.c:682
#5  0xc06a1e71 in kern_getpeername (td=0xc5971240, fd=7, sa=0xe9ae9c70, 
alen=0xe9ae9c6c)
    at /usr/src/sys/kern/uipc_syscalls.c:1566
#6  0xc06a1f72 in getpeername (td=0xc5971240, uap=0xe9ae9cf8)
    at /usr/src/sys/kern/uipc_syscalls.c:1527
#7  0xc0872415 in syscall (frame=0xe9ae9d38) at 
/usr/src/sys/i386/i386/trap.c:1073
#8  0xc0856970 in Xint0x80_syscall () at 
/usr/src/sys/i386/i386/exception.s:261
#9  0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)

-- 
Mel
From julian at elischer.org  Wed Jun 17 21:02:26 2009
From: julian at elischer.org (Julian Elischer)
Date: Wed Jun 17 21:02:32 2009
Subject: small usr.bin/find patch
In-Reply-To: <permail-200906171855491e86ffa800003434-a_best01@message-id.uni-muenster.de>
References: <permail-200906171855491e86ffa800003434-a_best01@message-id.uni-muenster.de>
Message-ID: <4A3959E1.1070306@elischer.org>

Alexander Best wrote:
> hi everybody,
> 
> here's just a quick hack i applied to find. i very often use the -size switch
> and always forget if you need to append "m" or "M" for megabyte or "k" or "K"
> for kilobyte. after applying the patch find accepts both. ;-)
> 
> cheers.
> 

Are you sure this is wise? after all 125 millibytes would be 1 bit..

(  :-)  )


> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

From jhb at freebsd.org  Wed Jun 17 21:18:11 2009
From: jhb at freebsd.org (John Baldwin)
Date: Wed Jun 17 21:18:18 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906171152.55438.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906170815.26650.jhb@freebsd.org>
	<200906171152.55438.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <200906171717.37677.jhb@freebsd.org>

On Wednesday 17 June 2009 3:52:54 pm Mel Flynn wrote:
> On Wednesday 17 June 2009 04:15:26 John Baldwin wrote:
> > On Tuesday 16 June 2009 7:01:45 pm Mel Flynn wrote:
> > > On Tuesday 16 June 2009 11:02:42 John Baldwin wrote:
> > > > On Tuesday 16 June 2009 1:52:23 pm Mel Flynn wrote:
> > > > > Hi John,
> > > > >
> > > > > On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> > > > > > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> > > > > > >   PID    TID COMM             TDNAME           KSTACK
> > > > > > >  4283 100215 kdeinit4         -                mi_switch
> > > > > > > turnstile_wait _mtx_lock_sleep uipc_peeraddr kern_getpeername
> > > > > > > getpeername syscall Xint0x80_syscall
> > > > > > > % ps -ww 4283
> > > > > > >   PID  TT  STAT      TIME COMMAND
> > > > > > >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > > > > > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket
> > > > > > > local:/tmp/ksocket- mel/plasmayC1653.slave-socket (kdeinit4)
> > > > > > >
> > > > > > > %ls -l /tmp/ksocket-mel/
> > > > > > >
> > > > > > > total 2
> > > > > > > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > > > > > > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > > > > > > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55
> > > > > > > klauncherxJ1635.slave-socket
> > > > > >
> > > > > > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb. 
> > > > > > Simply run 'kgdb' as root and do 'lcd /folder/with/scripts' and
> > > > > > 'source gdb6'. You can then do 'lockchain 4283' to find who holds
> > > > > > the lock this thread is blocked on and what state they are in.
> > > > >
> > > > > Looks like a deadlock:
> > > > >
> > > > > (kgdb) lockchain 4283
> > > > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > > 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on
> > > > > lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) 
blocked
> > > > > on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4)
> > > > > blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635,
> > > > > klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid
> > > > > 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122
> > > > > (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread
> > > > > 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > > > > thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348
> > > > > "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock
> > > > > 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on
> > > > > lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked
> > > > > on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher)
> > > > > blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283,
> > > > > kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid
> > > > > 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215
> > > > > (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread
> > > > > 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > > > > thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > > 0xc6806348 "unp_mtx" DEADLOCK
> > > > >
> > > > > Looking through the scripts now to see how I can get more info on 
the
> > > > > call chain and hoping I don't panic the machine ;). It is quite
> > > > > random to reproduce.
> > > >
> > > > In kgdb you can simply do 'tid 100122' followed by 'bt' and 'tid
> > > > 100215' followed by 'bt'.
> > >
> > > Cool, thanks for helping John. Of course it pretty much shows me what
> > > procstat -k shows and can't get any info on the userland part, but I can
> > > fully inspect the locks and threads.
> > >
> > > Both threads are in TDS_INHIBITED state, and blocked on:
> > > (kgdb) frame 0
> > > #0  sched_switch (td=0xc5971240, newtd=0xc4d39900, flags=259)
> > >     at /usr/src/sys/kern/sched_ule.c:1864
> > > 1864                    cpuid = PCPU_GET(cpuid);
> >
> > That doesn't really tell us anything except that it isn't running.  We 
know
> > it is actually blocked on a lock, and we need the full stack trace to see
> > where the two threads were trying to acquire the locks, hence 'bt'.  '
> > procstat -k' output would be fine, too.
> 
> the respective bt's:
> (kgdb) tid 100122
>     at /usr/src/sys/kern/kern_mutex.c:447
> #4  0xc06a68a5 in uipc_peeraddr (so=0xc64309a8, nam=0xe79a2c70)
>     at /usr/src/sys/kern/uipc_usrreq.c:682
> #5  0xc06a1e71 in kern_getpeername (td=0xc56e8900, fd=12, sa=0xe79a2c70, 
> alen=0xe79a2c6c)
>     at /usr/src/sys/kern/uipc_syscalls.c:1566
> 
> (kgdb) tid 100215
> (kgdb) bt
>     at /usr/src/sys/kern/kern_mutex.c:447
> #4  0xc06a68a5 in uipc_peeraddr (so=0xc6976338, nam=0xe9ae9c70)
>     at /usr/src/sys/kern/uipc_usrreq.c:682
> #5  0xc06a1e71 in kern_getpeername (td=0xc5971240, fd=7, sa=0xe9ae9c70, 
> alen=0xe9ae9c6c)
>     at /usr/src/sys/kern/uipc_syscalls.c:1566

These are the key frames.  It looks like uipc_peeraddr() tries to lock two unp 
locks w/o any protection from the global unp linkage lock.  I've changed it 
to use the same locking as uipc_accept() where it first grabs a read lock on 
the linkage lock and then just locks the other end of the connection to copy 
out its sockaddr.

--- //depot/user/jhb/socket/kern/uipc_usrreq.c
+++ /home/jhb/work/p4/socket/kern/uipc_usrreq.c
@@ -671,7 +671,7 @@
 	KASSERT(unp != NULL, ("uipc_peeraddr: unp == NULL"));
 
 	*nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK);
-	UNP_PCB_LOCK(unp);
+	UNP_LINK_RLOCK();
 	/*
 	 * XXX: It seems that this test always fails even when connection is
 	 * established.  So, this else clause is added as workaround to
@@ -681,7 +681,7 @@
 	if (unp2 != NULL) {
 		UNP_PCB_LOCK(unp2);
 		if (unp2->unp_addr != NULL)
-			sa = (struct sockaddr *) unp->unp_conn->unp_addr;
+			sa = (struct sockaddr *) unp2->unp_addr;
 		else
 			sa = &sun_noname;
 		bcopy(sa, *nam, sa->sa_len);
@@ -690,7 +690,7 @@
 		sa = &sun_noname;
 		bcopy(sa, *nam, sa->sa_len);
 	}
-	UNP_PCB_UNLOCK(unp);
+	UNP_LINK_RUNLOCK();
 	return (0);
 }
 
@@ -850,7 +850,7 @@
 		 * return the slightly counter-intuitive but otherwise
 		 * correct error that the socket is not connected.
 		 *
-		 * Locking here must be done carefully: the inkage lock
+		 * Locking here must be done carefully: the linkage lock
 		 * prevents interconnections between unpcbs from changing, so
 		 * we can traverse from unp to unp2 without acquiring unp's
 		 * lock.  Socket buffer locks follow unpcb locks, so we can

-- 
John Baldwin
From wojtek at wojtek.tensor.gdynia.pl  Wed Jun 17 21:26:31 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Wed Jun 17 21:26:37 2009
Subject: small usr.bin/find patch
In-Reply-To: <4A3959E1.1070306@elischer.org>
References: <permail-200906171855491e86ffa800003434-a_best01@message-id.uni-muenster.de>
	<4A3959E1.1070306@elischer.org>
Message-ID: <alpine.BSF.2.00.0906172325310.36849@wojtek.tensor.gdynia.pl>

>> 
>> cheers.
>> 
>
> Are you sure this is wise? after all 125 millibytes would be 1 bit..

Agree. While lots of people use m instead of M and b instead of B, this is 
not right to correct proper behavior to improper just for them.
From mel.flynn+fbsd.hackers at mailing.thruhere.net  Wed Jun 17 22:11:45 2009
From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn)
Date: Wed Jun 17 22:11:53 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906171717.37677.jhb@freebsd.org>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906171152.55438.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906171717.37677.jhb@freebsd.org>
Message-ID: <200906171411.42957.mel.flynn+fbsd.hackers@mailing.thruhere.net>

On Wednesday 17 June 2009 13:17:37 John Baldwin wrote:
> On Wednesday 17 June 2009 3:52:54 pm Mel Flynn wrote:
> > On Wednesday 17 June 2009 04:15:26 John Baldwin wrote:
> > > On Tuesday 16 June 2009 7:01:45 pm Mel Flynn wrote:
> > > > On Tuesday 16 June 2009 11:02:42 John Baldwin wrote:
> > > > > On Tuesday 16 June 2009 1:52:23 pm Mel Flynn wrote:
> > > > > > Hi John,
> > > > > >
> > > > > > On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> > > > > > > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> > > > > > > >   PID    TID COMM             TDNAME           KSTACK
> > > > > > > >  4283 100215 kdeinit4         -                mi_switch
> > > > > > > > turnstile_wait _mtx_lock_sleep uipc_peeraddr kern_getpeername
> > > > > > > > getpeername syscall Xint0x80_syscall
> > > > > > > > % ps -ww 4283
> > > > > > > >   PID  TT  STAT      TIME COMMAND
> > > > > > > >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > > > > > > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket
> > > > > > > > local:/tmp/ksocket- mel/plasmayC1653.slave-socket (kdeinit4)
> > > > > > > >
> > > > > > > > %ls -l /tmp/ksocket-mel/
> > > > > > > >
> > > > > > > > total 2
> > > > > > > > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > > > > > > > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > > > > > > > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55
> > > > > > > > klauncherxJ1635.slave-socket
> > > > > > >
> > > > > > > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb.
> > > > > > > Simply run 'kgdb' as root and do 'lcd /folder/with/scripts' and
> > > > > > > 'source gdb6'. You can then do 'lockchain 4283' to find who
> > > > > > > holds the lock this thread is blocked on and what state they
> > > > > > > are in.
> > > > > >
> > > > > > Looks like a deadlock:
> > > > > >
> > > > > > (kgdb) lockchain 4283
> > > > > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > > > 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked
> > > > > > on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher)
>
> blocked
>
> > > > > > on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4)
> > > > > > blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635,
> > > > > > klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215
> > > > > > (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread
> > > > > > 100122 (pid 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx"
> > > > > > thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > > > 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked
> > > > > > on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher)
> > > > > > blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283,
> > > > > > kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid
> > > > > > 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread
> > > > > > 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > > > > > thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348
> > > > > > "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock
> > > > > > 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked
> > > > > > on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4)
> > > > > > blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635,
> > > > > > klauncher) blocked on lock 0xc6806348 "unp_mtx" DEADLOCK
> > > > > >
> > > > > > Looking through the scripts now to see how I can get more info on
>
> the
>
> > > > > > call chain and hoping I don't panic the machine ;). It is quite
> > > > > > random to reproduce.
> > > > >
> > > > > In kgdb you can simply do 'tid 100122' followed by 'bt' and 'tid
> > > > > 100215' followed by 'bt'.
> > > >
> > > > Cool, thanks for helping John. Of course it pretty much shows me what
> > > > procstat -k shows and can't get any info on the userland part, but I
> > > > can fully inspect the locks and threads.
> > > >
> > > > Both threads are in TDS_INHIBITED state, and blocked on:
> > > > (kgdb) frame 0
> > > > #0  sched_switch (td=0xc5971240, newtd=0xc4d39900, flags=259)
> > > >     at /usr/src/sys/kern/sched_ule.c:1864
> > > > 1864                    cpuid = PCPU_GET(cpuid);
> > >
> > > That doesn't really tell us anything except that it isn't running.  We
>
> know
>
> > > it is actually blocked on a lock, and we need the full stack trace to
> > > see where the two threads were trying to acquire the locks, hence 'bt'.
> > >  ' procstat -k' output would be fine, too.
> >
> > the respective bt's:
> > (kgdb) tid 100122
> >     at /usr/src/sys/kern/kern_mutex.c:447
> > #4  0xc06a68a5 in uipc_peeraddr (so=0xc64309a8, nam=0xe79a2c70)
> >     at /usr/src/sys/kern/uipc_usrreq.c:682
> > #5  0xc06a1e71 in kern_getpeername (td=0xc56e8900, fd=12, sa=0xe79a2c70,
> > alen=0xe79a2c6c)
> >     at /usr/src/sys/kern/uipc_syscalls.c:1566
> >
> > (kgdb) tid 100215
> > (kgdb) bt
> >     at /usr/src/sys/kern/kern_mutex.c:447
> > #4  0xc06a68a5 in uipc_peeraddr (so=0xc6976338, nam=0xe9ae9c70)
> >     at /usr/src/sys/kern/uipc_usrreq.c:682
> > #5  0xc06a1e71 in kern_getpeername (td=0xc5971240, fd=7, sa=0xe9ae9c70,
> > alen=0xe9ae9c6c)
> >     at /usr/src/sys/kern/uipc_syscalls.c:1566
>
> These are the key frames.  It looks like uipc_peeraddr() tries to lock two
> unp locks w/o any protection from the global unp linkage lock.  I've
> changed it to use the same locking as uipc_accept() where it first grabs a
> read lock on the linkage lock and then just locks the other end of the
> connection to copy out its sockaddr.

Thanks John. I'll recompile the kernel with patch and up-to-date current and 
report back if there are any side effects or if the bug resurfaces.
Is there a sure way (i.e. testcase) that would expose this condition? At 
present, all I can do is wait and maybe play with network interface link 
up/down, as it seems to be related from a high level view.

-- 
Mel
From rwatson at FreeBSD.org  Wed Jun 17 22:32:44 2009
From: rwatson at FreeBSD.org (Robert Watson)
Date: Wed Jun 17 22:32:51 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906171717.37677.jhb@freebsd.org>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906170815.26650.jhb@freebsd.org>
	<200906171152.55438.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906171717.37677.jhb@freebsd.org>
Message-ID: <alpine.BSF.2.00.0906172332010.16422@fledge.watson.org>


On Wed, 17 Jun 2009, John Baldwin wrote:

> These are the key frames.  It looks like uipc_peeraddr() tries to lock two 
> unp locks w/o any protection from the global unp linkage lock.  I've changed 
> it to use the same locking as uipc_accept() where it first grabs a read lock 
> on the linkage lock and then just locks the other end of the connection to 
> copy out its sockaddr.

This change looks reasonable to me, thanks for tracking this down!

Robert N M Watson
Computer Laboratory
University of Cambridge

>
> --- //depot/user/jhb/socket/kern/uipc_usrreq.c
> +++ /home/jhb/work/p4/socket/kern/uipc_usrreq.c
> @@ -671,7 +671,7 @@
> 	KASSERT(unp != NULL, ("uipc_peeraddr: unp == NULL"));
>
> 	*nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK);
> -	UNP_PCB_LOCK(unp);
> +	UNP_LINK_RLOCK();
> 	/*
> 	 * XXX: It seems that this test always fails even when connection is
> 	 * established.  So, this else clause is added as workaround to
> @@ -681,7 +681,7 @@
> 	if (unp2 != NULL) {
> 		UNP_PCB_LOCK(unp2);
> 		if (unp2->unp_addr != NULL)
> -			sa = (struct sockaddr *) unp->unp_conn->unp_addr;
> +			sa = (struct sockaddr *) unp2->unp_addr;
> 		else
> 			sa = &sun_noname;
> 		bcopy(sa, *nam, sa->sa_len);
> @@ -690,7 +690,7 @@
> 		sa = &sun_noname;
> 		bcopy(sa, *nam, sa->sa_len);
> 	}
> -	UNP_PCB_UNLOCK(unp);
> +	UNP_LINK_RUNLOCK();
> 	return (0);
> }
>
> @@ -850,7 +850,7 @@
> 		 * return the slightly counter-intuitive but otherwise
> 		 * correct error that the socket is not connected.
> 		 *
> -		 * Locking here must be done carefully: the inkage lock
> +		 * Locking here must be done carefully: the linkage lock
> 		 * prevents interconnections between unpcbs from changing, so
> 		 * we can traverse from unp to unp2 without acquiring unp's
> 		 * lock.  Socket buffer locks follow unpcb locks, so we can
>
> -- 
> John Baldwin
>
From brooks at freebsd.org  Thu Jun 18 00:03:57 2009
From: brooks at freebsd.org (Brooks Davis)
Date: Thu Jun 18 00:04:10 2009
Subject: CFT: final patches for NGROUPS>>16
Message-ID: <20090618000407.GA43514@lor.one-eyed-alien.net>

Please find attached three patches which result in raising NGROUPS to
1024, making programs in the base system immune to changing values of
NGROUPS, and pave the way for NGROUPS to be boot time configurable.

The first two patches (ngroups-catman.diff and ngroups-posix.diff)
are userland cleanups.  The third (ngroups-main.diff) is the core of
the change and primairly in the kernel.  It is strongly based on work
contributed by Isilon Systems.  The proposed commit messages appear to
the bottom of this message.

I've been running the basic code on my FreeBSD laptop for a while and
I've tested basic NFS mounts, but more areas need testing before this
hits a release.  Key things to hit include:
 - NFS mount and export
 - IPFW uid, gid, and jail rules
 - portalfs (ideally portalfs should be extended to not truncate groups)

In practice, expect appliations to not care about this change unless
they are run with more than 16 groups.  If that happens, calls to
getgroups() will fail if they use statically sized buffers recompilation
should fix them.

Please test, review, etc.  I'd like to get this in before code freeze if
at all possible.

-- Brooks

ngroups-catman.diff:

  When checking if we can write to a file, use access() instead of a
  manual permission check based on stat output.  Also, get rid of the
  executability check since it is not used.

  MFC after:	2 weeks

ngroups-posix.diff

  In preparation for raising NGROUPS and NGROUPS_MAX, change base
  system callers of getgroups(), getgrouplist(), and setgroups() to
  allocate buffers dynamically.  Specifically, allocate a buffer of size
  sysconf(_SC_NGROUPS_MAX)+1 (+2 in a few cases to allow for overflow).

  This (or similar gymnastics) is required for the code to actually follow
  the POSIX.1-2008 specification where {NGROUPS_MAX} may differ at runtime
  and where getgroups may return {NGROUPS_MAX}+1 results on systems like
  FreeBSD which include the primary group.

  In id(1), don't pointlessly add the primary group to the list of all
  groups, it is always the first result from getgroups().  In principle
  the old code was more portable, but this was only done in one of the two
  places where getgroups() was called to the overall effect was pointless.

  Document the actual POSIX requirements in the getgroups(2) and
  setgroups(2) manpages.  We do not yet support a dynamic NGROUPS, but we
  may in the future.

  MFC after:	2 weeks

ngroups-main.diff:

  Rework the credential code to support larger values of NGROUPS and
  NGROUPS_MAX, eliminate ABI dependencies on them, and raise the to 1024
  and 1023 respectively.  (Previously they were equal, but under a close
  reading of POSIX, NGROUPS_MAX was defined to be too large by 1 since it
  is the number of supplemental groups, not total number of groups.)

  The bulk of the change consists of converting the struct ucred member
  cr_groups from a static array to a pointer.  Do the equivalent in
  kinfo_proc.

  Introduce new interfaces crcopysafe() and crsetgroups() for duplicating
  a process credential before modifying it and for setting group lists
  respectively.  Both interfaces take care for the details of allocating
  groups array. crsetgroups() takes care of truncating the group list
  to the current maximum (NGROUPS) if necessary.  In the future,
  crsetgroups() may be responsible for insuring invariants such as sorting
  the supplemental groups to allow groupmember() to be implemented as a
  binary search.

  Because we can not change struct xucred without breaking application
  ABIs, we leave it alone and introduce a new XU_NGROUPS value which is
  always 16 and is to be used or NGRPS as appropriate for things such as
  NFS which need to use no more than 16 groups.  When feasible, truncate
  the group list rather than generating an error.

  Minor changes:
    - Reduce the number of hand rolled versions of groupmember().
    - Do not assign to both cr_gid and cr_groups[0].
    - Modify ipfw to cache ucreds instead of part of their contents since
      they are immutable once referenced by more than one entity.

  Submitted by:	Isilon Systems
  X-MFC after:	never
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090618/ecdbb953/attachment.pgp
From brooks at freebsd.org  Thu Jun 18 00:05:21 2009
From: brooks at freebsd.org (Brooks Davis)
Date: Thu Jun 18 00:05:39 2009
Subject: CFT: final patches for NGROUPS>>16
In-Reply-To: <20090618000407.GA43514@lor.one-eyed-alien.net>
References: <20090618000407.GA43514@lor.one-eyed-alien.net>
Message-ID: <20090618000531.GA46033@lor.one-eyed-alien.net>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090618/564af13a/attachment-0001.pgp
From yuri at rawbw.com  Thu Jun 18 00:19:34 2009
From: yuri at rawbw.com (Yuri)
Date: Thu Jun 18 00:19:40 2009
Subject: valgrind on FreeBSD 7
In-Reply-To: <20081002073454.GA30869@nexus.in-nomine.org>
References: <200803172156.37407.modelnine@modelnine.org>	<20080317214510.G89676@odysseus.silby.com>	<20080413103351.GA1382@dose.local.invalid>	<4803C3B4.4000405@delphij.net>
	<48E474C7.8050507@rawbw.com>
	<20081002073454.GA30869@nexus.in-nomine.org>
Message-ID: <4A398783.5060806@rawbw.com>

Jeroen Ruigrok van der Werven wrote:
> I have been working on/off on it.
>
> I am trying to find my work in progress sources, but I think they got lost
> when a hard disk died.
>
> This is all I have found:
>
> http://www.in-nomine.org/~asmodai/valgrind/valgrind-trunk-for-freebsd.diff
>   


Any news on valgrind for FreeBSD?
Also is there a way to check it out from the server?
Is it on perforce? I believe there is no easy way to even get it 
read-only without user id.
So are there snapshots somewhere?

Yuri

From elias at artx.ru  Thu Jun 18 06:18:13 2009
From: elias at artx.ru (Ilya Orehov)
Date: Thu Jun 18 06:18:21 2009
Subject: small usr.bin/find patch
In-Reply-To: <4A3959E1.1070306@elischer.org>
References: <permail-200906171855491e86ffa800003434-a_best01@message-id.uni-muenster.de>
	<4A3959E1.1070306@elischer.org>
Message-ID: <20090618061810.GA11636@artx.ru>

+------- Julian Elischer, 2009-06-17 -------
| Alexander Best wrote:
| >hi everybody,
| >
| >here's just a quick hack i applied to find. i very often use the -size 
| >switch
| >and always forget if you need to append "m" or "M" for megabyte or "k" or 
| >"K"
| >for kilobyte. after applying the patch find accepts both. ;-)
| >
| >cheers.
| >
| 
| Are you sure this is wise? after all 125 millibytes would be 1 bit..
| 
| (  :-)  )

BTW, our du(1) accepts only lowercase -k and -m.
From malathiramya at gmail.com  Thu Jun 18 06:24:00 2009
From: malathiramya at gmail.com (malathi selvaraj)
Date: Thu Jun 18 06:24:07 2009
Subject: good morning to all
Message-ID: <fc09aacb0906172323u56f0d93cg77e5be78fe3e2b22@mail.gmail.com>

how to  install samba in freebsd
advance thanks
From ttw+bsd at cobbled.net  Thu Jun 18 07:43:27 2009
From: ttw+bsd at cobbled.net (ttw+bsd@cobbled.net)
Date: Thu Jun 18 07:43:40 2009
Subject: CFT: final patches for NGROUPS>>16
In-Reply-To: <20090618000407.GA43514@lor.one-eyed-alien.net>
References: <20090618000407.GA43514@lor.one-eyed-alien.net>
Message-ID: <20090618071643.GA27928@holyman.cobbled.net>

posted patches to this effect some months ago.  they needed some
clean-up and validation but the also mapped correctly into an
RPC_NGROUPS_MAX and IPC_NGROUPS_MAX for consistent (and possibly
dynamic) mapping to those problem areas.  they build and run but are
untested beyond that.  i do not expect to have time to re-visit the
problem in reasonable order.  they also included some other userland
changes that may be of interest.
From des at des.no  Thu Jun 18 07:57:38 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Thu Jun 18 07:57:45 2009
Subject: valgrind on FreeBSD 7
In-Reply-To: <4A398783.5060806@rawbw.com> (yuri@rawbw.com's message of "Wed,
	17 Jun 2009 17:17:07 -0700")
References: <200803172156.37407.modelnine@modelnine.org>
	<20080317214510.G89676@odysseus.silby.com>
	<20080413103351.GA1382@dose.local.invalid>
	<4803C3B4.4000405@delphij.net> <48E474C7.8050507@rawbw.com>
	<20081002073454.GA30869@nexus.in-nomine.org>
	<4A398783.5060806@rawbw.com>
Message-ID: <86eitidle6.fsf@ds4.des.no>

Yuri <yuri@rawbw.com> writes:
> Any news on valgrind for FreeBSD?

http://perforce.freebsd.org/changeList.cgi?FSPC=//depot/projects/valgrind/...

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From malathiramya at gmail.com  Thu Jun 18 09:59:39 2009
From: malathiramya at gmail.com (malathi selvaraj)
Date: Thu Jun 18 09:59:46 2009
Subject: freebsd-hackers Digest, Vol 325, Issue 4
In-Reply-To: <20090618000546.433BB10657B5@hub.freebsd.org>
References: <20090618000546.433BB10657B5@hub.freebsd.org>
Message-ID: <fc09aacb0906180259j3368cafbl7c06083479e3d3f1@mail.gmail.com>

how to start local host is freeBSD,
i install apache22 even after localhost is not working
From jhb at freebsd.org  Thu Jun 18 13:37:40 2009
From: jhb at freebsd.org (John Baldwin)
Date: Thu Jun 18 13:39:52 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906171411.42957.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906171717.37677.jhb@freebsd.org>
	<200906171411.42957.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <200906180923.57749.jhb@freebsd.org>

On Wednesday 17 June 2009 6:11:42 pm Mel Flynn wrote:
> On Wednesday 17 June 2009 13:17:37 John Baldwin wrote:
> > On Wednesday 17 June 2009 3:52:54 pm Mel Flynn wrote:
> > > On Wednesday 17 June 2009 04:15:26 John Baldwin wrote:
> > > > On Tuesday 16 June 2009 7:01:45 pm Mel Flynn wrote:
> > > > > On Tuesday 16 June 2009 11:02:42 John Baldwin wrote:
> > > > > > On Tuesday 16 June 2009 1:52:23 pm Mel Flynn wrote:
> > > > > > > Hi John,
> > > > > > >
> > > > > > > On Tuesday 16 June 2009 04:19:57 John Baldwin wrote:
> > > > > > > > On Monday 15 June 2009 5:53:05 pm Mel Flynn wrote:
> > > > > > > > >   PID    TID COMM             TDNAME           KSTACK
> > > > > > > > >  4283 100215 kdeinit4         -                mi_switch
> > > > > > > > > turnstile_wait _mtx_lock_sleep uipc_peeraddr 
kern_getpeername
> > > > > > > > > getpeername syscall Xint0x80_syscall
> > > > > > > > > % ps -ww 4283
> > > > > > > > >   PID  TT  STAT      TIME COMMAND
> > > > > > > > >  4283  ??  T      0:00.38 kdeinit4: kdeinit4: kio_http http
> > > > > > > > > local:/tmp/ksocket-mel/klauncherxJ1635.slave-socket
> > > > > > > > > local:/tmp/ksocket- mel/plasmayC1653.slave-socket (kdeinit4)
> > > > > > > > >
> > > > > > > > > %ls -l /tmp/ksocket-mel/
> > > > > > > > >
> > > > > > > > > total 2
> > > > > > > > > -rw-rw-r--  1 mel  wheel  62 Jun 14 22:55 KSMserver__0
> > > > > > > > > srw-------  1 mel  wheel   0 Jun 14 22:55 kdeinit4__0
> > > > > > > > > srwxrwxr-x  1 mel  wheel   0 Jun 14 22:55
> > > > > > > > > klauncherxJ1635.slave-socket
> > > > > > > >
> > > > > > > > You can use kgdb and the scripts at www.freebsd.org/~jhb/gdb.
> > > > > > > > Simply run 'kgdb' as root and do 'lcd /folder/with/scripts' 
and
> > > > > > > > 'source gdb6'. You can then do 'lockchain 4283' to find who
> > > > > > > > holds the lock this thread is blocked on and what state they
> > > > > > > > are in.
> > > > > > >
> > > > > > > Looks like a deadlock:
> > > > > > >
> > > > > > > (kgdb) lockchain 4283
> > > > > > >  thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > > > > 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked
> > > > > > > on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher)
> >
> > blocked
> >
> > > > > > > on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4)
> > > > > > > blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635,
> > > > > > > klauncher) blocked on lock 0xc6806348 "unp_mtx" thread 100215
> > > > > > > (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread
> > > > > > > 100122 (pid 1635, klauncher) blocked on lock 
0xc6806348 "unp_mtx"
> > > > > > > thread 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0
> > > > > > > "unp_mtx" thread 100122 (pid 1635, klauncher) blocked on lock
> > > > > > > 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked
> > > > > > > on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher)
> > > > > > > blocked on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283,
> > > > > > > kdeinit4) blocked on lock 0xc64374a0 "unp_mtx" thread 100122 
(pid
> > > > > > > 1635, klauncher) blocked on lock 0xc6806348 "unp_mtx" thread
> > > > > > > 100215 (pid 4283, kdeinit4) blocked on lock 0xc64374a0 "unp_mtx"
> > > > > > > thread 100122 (pid 1635, klauncher) blocked on lock 0xc6806348
> > > > > > > "unp_mtx" thread 100215 (pid 4283, kdeinit4) blocked on lock
> > > > > > > 0xc64374a0 "unp_mtx" thread 100122 (pid 1635, klauncher) blocked
> > > > > > > on lock 0xc6806348 "unp_mtx" thread 100215 (pid 4283, kdeinit4)
> > > > > > > blocked on lock 0xc64374a0 "unp_mtx" thread 100122 (pid 1635,
> > > > > > > klauncher) blocked on lock 0xc6806348 "unp_mtx" DEADLOCK
> > > > > > >
> > > > > > > Looking through the scripts now to see how I can get more info 
on
> >
> > the
> >
> > > > > > > call chain and hoping I don't panic the machine ;). It is quite
> > > > > > > random to reproduce.
> > > > > >
> > > > > > In kgdb you can simply do 'tid 100122' followed by 'bt' and 'tid
> > > > > > 100215' followed by 'bt'.
> > > > >
> > > > > Cool, thanks for helping John. Of course it pretty much shows me 
what
> > > > > procstat -k shows and can't get any info on the userland part, but I
> > > > > can fully inspect the locks and threads.
> > > > >
> > > > > Both threads are in TDS_INHIBITED state, and blocked on:
> > > > > (kgdb) frame 0
> > > > > #0  sched_switch (td=0xc5971240, newtd=0xc4d39900, flags=259)
> > > > >     at /usr/src/sys/kern/sched_ule.c:1864
> > > > > 1864                    cpuid = PCPU_GET(cpuid);
> > > >
> > > > That doesn't really tell us anything except that it isn't running.  We
> >
> > know
> >
> > > > it is actually blocked on a lock, and we need the full stack trace to
> > > > see where the two threads were trying to acquire the locks, 
hence 'bt'.
> > > >  ' procstat -k' output would be fine, too.
> > >
> > > the respective bt's:
> > > (kgdb) tid 100122
> > >     at /usr/src/sys/kern/kern_mutex.c:447
> > > #4  0xc06a68a5 in uipc_peeraddr (so=0xc64309a8, nam=0xe79a2c70)
> > >     at /usr/src/sys/kern/uipc_usrreq.c:682
> > > #5  0xc06a1e71 in kern_getpeername (td=0xc56e8900, fd=12, sa=0xe79a2c70,
> > > alen=0xe79a2c6c)
> > >     at /usr/src/sys/kern/uipc_syscalls.c:1566
> > >
> > > (kgdb) tid 100215
> > > (kgdb) bt
> > >     at /usr/src/sys/kern/kern_mutex.c:447
> > > #4  0xc06a68a5 in uipc_peeraddr (so=0xc6976338, nam=0xe9ae9c70)
> > >     at /usr/src/sys/kern/uipc_usrreq.c:682
> > > #5  0xc06a1e71 in kern_getpeername (td=0xc5971240, fd=7, sa=0xe9ae9c70,
> > > alen=0xe9ae9c6c)
> > >     at /usr/src/sys/kern/uipc_syscalls.c:1566
> >
> > These are the key frames.  It looks like uipc_peeraddr() tries to lock two
> > unp locks w/o any protection from the global unp linkage lock.  I've
> > changed it to use the same locking as uipc_accept() where it first grabs a
> > read lock on the linkage lock and then just locks the other end of the
> > connection to copy out its sockaddr.
> 
> Thanks John. I'll recompile the kernel with patch and up-to-date current and 
> report back if there are any side effects or if the bug resurfaces.
> Is there a sure way (i.e. testcase) that would expose this condition? At 
> present, all I can do is wait and maybe play with network interface link 
> up/down, as it seems to be related from a high level view.

It looks like two ends of a PF_LOCAL socket doing getpeername() at the same 
time would provoke this.  I can maybe write a test program today to provoke 
this and test it locally.

-- 
John Baldwin
From attilio at freebsd.org  Thu Jun 18 15:13:54 2009
From: attilio at freebsd.org (Attilio Rao)
Date: Thu Jun 18 15:14:01 2009
Subject: valgrind on FreeBSD 7
In-Reply-To: <48E474C7.8050507@rawbw.com>
References: <200803172156.37407.modelnine@modelnine.org>
	<20080317214510.G89676@odysseus.silby.com>
	<20080413103351.GA1382@dose.local.invalid>
	<4803C3B4.4000405@delphij.net> <48E474C7.8050507@rawbw.com>
Message-ID: <3bbf2fe10906180743pc9bd84bu420db38275fe3888@mail.gmail.com>

2008/10/2 Yuri <yuri@rawbw.com>:
> Xin LI wrote:
>>
>> Simon Barner wrote:
>>>
>>> Mike Silbersack wrote:
>>>>
>>>> On Mon, 17 Mar 2008, Heiko Wundram wrote:
>>>>
>>>> Here's a tarball of what's in perforce right now.  I tried it a little
>>>> bit, and it seemed to work for me.  Make sure to install the kernel module!
>>>>
>>>> http://www.silby.com/valgrind_freebsd_3.tar.gz
>>>>
>>>> But don't send me questions about it - I'm not an expert on it, I'm just
>>>> the guy who grabbed it from perforce and found that it seems to work. :)
>>>
>>> Could you please provide me with the details, so I can update my
>>> (horribly outdated :( valgrind ports?
>>
>> It was available from p4 at:
>>
>> //depot/projects/valgrind/...
>>
>> Note that this version does not work on architectures other than i386.
>>
>> Cheers,
>
>
> Any developments in Valgrind/Callgrind on FreeBSD?
> Any hope to get this version into ports and to merge FreeBSD support up into
> Valgrind project?

For our employer (Sandvine incorporated) Ed Maste and me are working
on finishing the port Peter Wemm started a while ago.
You can find updated informations on the project here:
http://wiki.freebsd.org/Valgrind

while you can get the sources pack here:
http://people.freebsd.org/~attilio/Sandvine/valgrind/valgrind.tar.bz2

Please note that 8 and 7 now have all the required support to let run
this modified version, so no further modify is required to these
branches.
Valgrind is able to run on 6 too, if the patch system_valgrind.diff is
applied (I can't recall if it targeted for SVOS, as I think, or it is
stock STABLE_6, just try it, it may apply to both).

The port is really not clean, but that's not Peter's fault. Valgrind
should be really rewritten in order to be less Linux-centric and be
more smart about interface sucking. The effort, though, would be so
large that would require too much work to happen in the short term.

What is in the tar works mostly ok (modulo missing syscalls) on both
ia32 and amd64, but you should carefully follow what is written in the
Wiki for a good install of the product.
Also note that I have an internal version that should fix most (all?)
the issues reported in the Wiki and I will try to release it on a
regular basis, until possible.
So far, I'm fighting with a segfault in the profiled process while
running with SV environment {amd64 arch}, a symptomancy that doesn't
show with a stock FreeBSD installation so far. If you could try the
port and report to us I would appreciate a lot.

For any other question please refer to us.

Thanks,
Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
From wojtek at wojtek.tensor.gdynia.pl  Thu Jun 18 18:58:22 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Thu Jun 18 18:58:28 2009
Subject: freebsd-hackers Digest, Vol 325, Issue 4
In-Reply-To: <fc09aacb0906180259j3368cafbl7c06083479e3d3f1@mail.gmail.com>
References: <20090618000546.433BB10657B5@hub.freebsd.org>
	<fc09aacb0906180259j3368cafbl7c06083479e3d3f1@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906182058050.43262@wojtek.tensor.gdynia.pl>

start reading books

On Thu, 18 Jun 2009, malathi selvaraj wrote:

> how to start local host is freeBSD,
> i install apache22 even after localhost is not working
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>
>
From jhb at freebsd.org  Thu Jun 18 20:54:09 2009
From: jhb at freebsd.org (John Baldwin)
Date: Thu Jun 18 20:54:16 2009
Subject: How best to debug locking/scheduler problems
In-Reply-To: <200906171411.42957.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <200906151353.06630.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<200906171717.37677.jhb@freebsd.org>
	<200906171411.42957.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <200906181654.02970.jhb@freebsd.org>

On Wednesday 17 June 2009 6:11:42 pm Mel Flynn wrote:
> On Wednesday 17 June 2009 13:17:37 John Baldwin wrote:
> > These are the key frames.  It looks like uipc_peeraddr() tries to lock two
> > unp locks w/o any protection from the global unp linkage lock.  I've
> > changed it to use the same locking as uipc_accept() where it first grabs a
> > read lock on the linkage lock and then just locks the other end of the
> > connection to copy out its sockaddr.
> 
> Thanks John. I'll recompile the kernel with patch and up-to-date current and 
> report back if there are any side effects or if the bug resurfaces.
> Is there a sure way (i.e. testcase) that would expose this condition? At 
> present, all I can do is wait and maybe play with network interface link 
> up/down, as it seems to be related from a high level view.

I write a testcase for this that had two threads calling getpeername() against 
each other in a loop.  It locked up on a stock 7.x box in a few seconds. :)  
It has run to completion without deadlocking with the patch.

-- 
John Baldwin
From des at des.no  Thu Jun 18 22:18:48 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Thu Jun 18 22:18:55 2009
Subject: svn commit: r193635 - head/etc
In-Reply-To: <4A3A88BD.5030703@FreeBSD.org> (Doug Barton's message of "Thu, 18
	Jun 2009 11:34:37 -0700")
References: <200906071326.n57DQvSG095104@svn.freebsd.org>
	<86iqiudm63.fsf@ds4.des.no> <4A3A88BD.5030703@FreeBSD.org>
Message-ID: <86ab455gop.fsf@ds4.des.no>

Doug Barton <dougb@FreeBSD.org> writes:
> Dag-Erling Sm?rgrav <des@des.no> writes:
> > Great, now mergemaster blew away my ntp.conf and installed this one
> > instead.  Apparently, it thinks AUTO_UPGRADE means it's fine to
> > overwrite an existing file with a new one...
> Yes, that's exactly what the option means. The problem comes in
> because it's a new file, which means that there is no record of it in
> the mtree file, so it does not show up as "changed."

Hmm, I'm not sure I follow, since I'm not familiar with the innards of
mergemaster, but can you tell it's new?  If you can, you can check if
there's already a file of the same name before installing the new one?

> FWIW, this is one of the reasons that I resisted the idea of using
> mtree for this function, and continue to resist the idea of the -U
> option being the default.

I didn't realize it was the default - but I really, really like it.  It
makes mergemaster a *lot* easier to use.

> There is no way that I can see to have mtree list the files that have
> _not_ changed, which would be the safest way to implement this
> option.

Doesn't sound unsurmountable.

> Meanwhile I'm sure you were able to restore from backups

Of course (not that there was much to restore - just "server ntp.des.no
iburst maxpoll 6"), and now that I know about it, I can list it in
IGNORE_FILES along with motd and printcap.

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From des at des.no  Thu Jun 18 23:47:44 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Thu Jun 18 23:54:32 2009
Subject: freebsd-hackers Digest, Vol 325, Issue 4
In-Reply-To: <fc09aacb0906180259j3368cafbl7c06083479e3d3f1@mail.gmail.com>
	(malathi selvaraj's message of "Thu, 18 Jun 2009 15:29:37 +0530")
References: <20090618000546.433BB10657B5@hub.freebsd.org>
	<fc09aacb0906180259j3368cafbl7c06083479e3d3f1@mail.gmail.com>
Message-ID: <86skhxnly9.fsf@ds4.des.no>

malathi selvaraj <malathiramya@gmail.com> writes:
> how to start local host is freeBSD,
> i install apache22 even after localhost is not working

You'll have to state your question more clearly...  and ask it on
-questions; it is not appropriate for this list.

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From des at des.no  Fri Jun 19 05:02:30 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Fri Jun 19 05:02:37 2009
Subject: svn commit: r193635 - head/etc
In-Reply-To: <4A3B1AD9.90507@FreeBSD.org> (Doug Barton's message of "Thu, 18
	Jun 2009 21:58:01 -0700")
References: <200906071326.n57DQvSG095104@svn.freebsd.org>
	<86iqiudm63.fsf@ds4.des.no> <4A3A88BD.5030703@FreeBSD.org>
	<86ab455gop.fsf@ds4.des.no> <4A3B1AD9.90507@FreeBSD.org>
Message-ID: <86k538oly2.fsf@ds4.des.no>

Doug Barton <dougb@FreeBSD.org> writes:
> Now that you've had a successful run with the new sources the
> signature for the stock file will be in mergemaster's mtree file so
> you won't have to worry.

I will for my other machines :)

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From dougb at FreeBSD.org  Fri Jun 19 05:24:48 2009
From: dougb at FreeBSD.org (Doug Barton)
Date: Fri Jun 19 05:24:56 2009
Subject: svn commit: r193635 - head/etc
In-Reply-To: <86ab455gop.fsf@ds4.des.no>
References: <200906071326.n57DQvSG095104@svn.freebsd.org>	<86iqiudm63.fsf@ds4.des.no>
	<4A3A88BD.5030703@FreeBSD.org> <86ab455gop.fsf@ds4.des.no>
Message-ID: <4A3B1AD9.90507@FreeBSD.org>

Dag-Erling Sm?rgrav wrote:
> Doug Barton <dougb@FreeBSD.org> writes:
>> Dag-Erling Sm?rgrav <des@des.no> writes:
>>> Great, now mergemaster blew away my ntp.conf and installed this one
>>> instead.  Apparently, it thinks AUTO_UPGRADE means it's fine to
>>> overwrite an existing file with a new one...
>> Yes, that's exactly what the option means. The problem comes in
>> because it's a new file, which means that there is no record of it in
>> the mtree file, so it does not show up as "changed."
> 
> Hmm, I'm not sure I follow, since I'm not familiar with the innards of
> mergemaster,

The -U option works by comparing the installed files to the mtree file
created from the unmodified source files. If the file is listed as
having been changed, the -U option ignores it. If not, it
auto-installs it.

> but can you tell it's new?  If you can, you can check if
> there's already a file of the same name before installing the new one?

The whole point of the option is to automatically overwrite the
existing file if it isn't listed as being changed.

>> FWIW, this is one of the reasons that I resisted the idea of using
>> mtree for this function, and continue to resist the idea of the -U
>> option being the default.
> 
> I didn't realize it was the default - but I really, really like it.  It
> makes mergemaster a *lot* easier to use.

It's not the default. Corner cases like this one are the reason I
resist making it the default.

>> There is no way that I can see to have mtree list the files that have
>> _not_ changed, which would be the safest way to implement this
>> option.
> 
> Doesn't sound unsurmountable.
> 
>> Meanwhile I'm sure you were able to restore from backups
> 
> Of course (not that there was much to restore - just "server ntp.des.no
> iburst maxpoll 6"), and now that I know about it, I can list it in
> IGNORE_FILES along with motd and printcap.

Now that you've had a successful run with the new sources the
signature for the stock file will be in mergemaster's mtree file so
you won't have to worry. You might also consider adding the svn Id for
the source file which will allow mergemaster to ignore it altogether
at least until it changes.


hope this helps,

Doug

-- 

    This .signature sanitized for your protection

From jhay at meraka.org.za  Fri Jun 19 05:33:52 2009
From: jhay at meraka.org.za (John Hay)
Date: Fri Jun 19 05:33:59 2009
Subject: svn commit: r193635 - head/etc
In-Reply-To: <4A3B1AD9.90507@FreeBSD.org>
References: <200906071326.n57DQvSG095104@svn.freebsd.org>
	<86iqiudm63.fsf@ds4.des.no> <4A3A88BD.5030703@FreeBSD.org>
	<86ab455gop.fsf@ds4.des.no> <4A3B1AD9.90507@FreeBSD.org>
Message-ID: <20090619053343.GA1374@zibbi.meraka.csir.co.za>

On Thu, Jun 18, 2009 at 09:58:01PM -0700, Doug Barton wrote:
> Dag-Erling Sm??rgrav wrote:
> > Doug Barton <dougb@FreeBSD.org> writes:
> >> Dag-Erling Sm??rgrav <des@des.no> writes:
> >>> Great, now mergemaster blew away my ntp.conf and installed this one
> >>> instead.  Apparently, it thinks AUTO_UPGRADE means it's fine to
> >>> overwrite an existing file with a new one...
> >> Yes, that's exactly what the option means. The problem comes in
> >> because it's a new file, which means that there is no record of it in
> >> the mtree file, so it does not show up as "changed."
> > 
> > Hmm, I'm not sure I follow, since I'm not familiar with the innards of
> > mergemaster,
> 
> The -U option works by comparing the installed files to the mtree file
> created from the unmodified source files. If the file is listed as
> having been changed, the -U option ignores it. If not, it
> auto-installs it.

Is it not possible to change the logic of -U a little. Only auto install
if it is in mtree and has not changed. So if it has changed or is not in
mtree, skip the auto install.

John
-- 
John Hay -- jhay@meraka.csir.co.za / jhay@FreeBSD.org
From dougb at FreeBSD.org  Fri Jun 19 05:36:14 2009
From: dougb at FreeBSD.org (Doug Barton)
Date: Fri Jun 19 05:36:20 2009
Subject: svn commit: r193635 - head/etc
In-Reply-To: <20090619053343.GA1374@zibbi.meraka.csir.co.za>
References: <200906071326.n57DQvSG095104@svn.freebsd.org>
	<86iqiudm63.fsf@ds4.des.no> <4A3A88BD.5030703@FreeBSD.org>
	<86ab455gop.fsf@ds4.des.no> <4A3B1AD9.90507@FreeBSD.org>
	<20090619053343.GA1374@zibbi.meraka.csir.co.za>
Message-ID: <4A3B23CA.4070701@FreeBSD.org>

John Hay wrote:
> Is it not possible to change the logic of -U a little. Only auto install
> if it is in mtree and has not changed. So if it has changed or is not in
> mtree, skip the auto install.

Apparently you didn't read the whole thread, but the answer is no.

Doug

-- 

    This .signature sanitized for your protection

From vasanth.raonaik at gmail.com  Fri Jun 19 12:38:51 2009
From: vasanth.raonaik at gmail.com (vasanth raonaik)
Date: Fri Jun 19 12:38:57 2009
Subject: Regarding Signal IPC
Message-ID: <d9f479c10906190512x4d1cf271w4107bfee0404e52a@mail.gmail.com>

Hello Hackers,

I want to print out the process ID of the process which is sending the
Signal.

Is it possible. if yes, can you please point me to any related documents.

Thanks,
Vasanth
From deischen at freebsd.org  Fri Jun 19 13:03:30 2009
From: deischen at freebsd.org (Daniel Eischen)
Date: Fri Jun 19 13:03:36 2009
Subject: Regarding Signal IPC
In-Reply-To: <d9f479c10906190512x4d1cf271w4107bfee0404e52a@mail.gmail.com>
References: <d9f479c10906190512x4d1cf271w4107bfee0404e52a@mail.gmail.com>
Message-ID: <Pine.GSO.4.64.0906190859550.9039@sea.ntplx.net>

On Fri, 19 Jun 2009, vasanth raonaik wrote:

> Hello Hackers,
>
> I want to print out the process ID of the process which is sending the
> Signal.
>
> Is it possible. if yes, can you please point me to any related documents.

Though I have not tried this, there is an si_pid field
(and other fields you might be interested in) in
struct siginfo.  If you use a POSIX signal handler
(see sigaction(2)), a pointer to a struct siginfo is
the 2nd argument to your signal handler.

See <sys/signal.h> for the definition of struct siginfo.

-- 
DE
From jilles at stack.nl  Fri Jun 19 14:03:55 2009
From: jilles at stack.nl (Jilles Tjoelker)
Date: Fri Jun 19 14:04:02 2009
Subject: Regarding Signal IPC
In-Reply-To: <Pine.GSO.4.64.0906190859550.9039@sea.ntplx.net>
References: <d9f479c10906190512x4d1cf271w4107bfee0404e52a@mail.gmail.com>
	<Pine.GSO.4.64.0906190859550.9039@sea.ntplx.net>
Message-ID: <20090619140326.GA75222@stack.nl>

On Fri, Jun 19, 2009 at 09:03:28AM -0400, Daniel Eischen wrote:
> On Fri, 19 Jun 2009, vasanth raonaik wrote:
> > I want to print out the process ID of the process which is sending the
> > Signal.

> > Is it possible. if yes, can you please point me to any related documents.

> Though I have not tried this, there is an si_pid field
> (and other fields you might be interested in) in
> struct siginfo.  If you use a POSIX signal handler
> (see sigaction(2)), a pointer to a struct siginfo is
> the 2nd argument to your signal handler.

> See <sys/signal.h> for the definition of struct siginfo.

Meaningful siginfo is only given for signals sent by sigqueue(2), traps,
child processes (SIGCHLD) and sigevent structures (for example
mq_notify(2)).

If you want to handle signals synchronously, you can use
sigtimedwait(2). It is still necessary to call sigaction(2) with the
SA_SIGINFO flag, even though the signal handler will not be called
because the signal is blocked (a necessary step before using
sigwait-like calls).

This alternate mechanism may be particularly useful here because getting
the siginfo data out of the signal handler safely is a tricky business.

Although I haven't tried this, kqueue's SIGEV_SIGNAL may be helpful in
managing this from a single-threaded program. This is, however, not
portable.

-- 
Jilles Tjoelker
From des at des.no  Fri Jun 19 20:14:17 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Fri Jun 19 20:14:24 2009
Subject: svn commit: r193635 - head/etc
In-Reply-To: <4A3B23CA.4070701@FreeBSD.org> (Doug Barton's message of "Thu, 18
	Jun 2009 22:36:10 -0700")
References: <200906071326.n57DQvSG095104@svn.freebsd.org>
	<86iqiudm63.fsf@ds4.des.no> <4A3A88BD.5030703@FreeBSD.org>
	<86ab455gop.fsf@ds4.des.no> <4A3B1AD9.90507@FreeBSD.org>
	<20090619053343.GA1374@zibbi.meraka.csir.co.za>
	<4A3B23CA.4070701@FreeBSD.org>
Message-ID: <867hz8nfqg.fsf@ds4.des.no>

Doug Barton <dougb@FreeBSD.org> writes:
> John Hay <jhay@meraka.org.za> writes:
> > Is it not possible to change the logic of -U a little. Only auto install
> > if it is in mtree and has not changed. So if it has changed or is not in
> > mtree, skip the auto install.
> Apparently you didn't read the whole thread, but the answer is no.

Or rather "not with the tools currently available in FreeBSD".

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From wojtek at wojtek.tensor.gdynia.pl  Sun Jun 21 15:16:04 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Sun Jun 21 15:16:10 2009
Subject: is RTL8139 THAT bad?
Message-ID: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>

i have pentium 200 with that card. doing ftp from other machine, getting 
3.5MB/s (HDD can 10MB/s, DMA) having 45%-55% interrupt load.

when sending it's not that bad.

tried writing file to disk with cat /dev/zero >file, it's only 3% ints 
with 10MB/s traffic.


Why it's THAT bad?

3.5MB/s is less that 2500 packets/second. 50% at 200Mhz means 100000000 
cycles spend on interrupt service, which is 40000 CPU cycles per packet.

From des at des.no  Sun Jun 21 15:46:15 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Sun Jun 21 15:46:22 2009
Subject: is RTL8139 THAT bad?
In-Reply-To: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
	(Wojciech Puchar's message of "Sun, 21 Jun 2009 17:14:29 +0200
	(CEST)")
References: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
Message-ID: <86eitdy4hl.fsf@ds4.des.no>

Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> writes:
> Why it's THAT bad?

http://svn.freebsd.org/base/head/sys/pci/if_rl.c

Scroll down past the copyright, license and attribution.  Read the
38-line comment that explains just how crappy this chip really is.

Executive summary: every single transmitted frame must be copied from
the mbuf into a DMA transmit buffer, and every single received frame
must be copied from the (quite small) DMA receive buffer into an mbuf.
In addition, the transmit queue can only hold four frames.

Other chips use scatter-gather lists and other mechanisms which allow
them to DMA frames straight out of or into mbufs.

DES
-- 
Dag-Erling Sm?rgrav - des@des.no
From wojtek at wojtek.tensor.gdynia.pl  Sun Jun 21 20:23:14 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Sun Jun 21 20:23:21 2009
Subject: is RTL8139 THAT bad?
In-Reply-To: <86eitdy4hl.fsf@ds4.des.no>
References: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
	<86eitdy4hl.fsf@ds4.des.no>
Message-ID: <alpine.BSF.2.00.0906212219190.77226@wojtek.tensor.gdynia.pl>

> Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> writes:
>> Why it's THAT bad?
>
> http://svn.freebsd.org/base/head/sys/pci/if_rl.c
>
> Scroll down past the copyright, license and attribution.  Read the
> 38-line comment that explains just how crappy this chip really is.

Well - really "low end".

But - this computer can do memcpy at 80MB/s, so at 3.5MB/s it should be 5% 
CPU for memcpy, and one interrupt per one packet (2500 packets/s).

Is something more that make it consume >50% CPU?
From pyunyh at gmail.com  Mon Jun 22 00:35:56 2009
From: pyunyh at gmail.com (Pyun YongHyeon)
Date: Mon Jun 22 00:36:03 2009
Subject: is RTL8139 THAT bad?
In-Reply-To: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
Message-ID: <20090622001705.GA10712@michelle.cdnetworks.co.kr>

On Sun, Jun 21, 2009 at 05:14:29PM +0200, Wojciech Puchar wrote:
> i have pentium 200 with that card. doing ftp from other machine, getting 
> 3.5MB/s (HDD can 10MB/s, DMA) having 45%-55% interrupt load.
> 
> when sending it's not that bad.
> 
> tried writing file to disk with cat /dev/zero >file, it's only 3% ints 
> with 10MB/s traffic.
> 
> 
> Why it's THAT bad?
> 

Because CPU always have to copy frames to/from the controller.
These CPU cycles could have been used in other task to give more
performance such as SSH encryption/decryption, checksum computation
etc.

> 3.5MB/s is less that 2500 packets/second. 50% at 200Mhz means 100000000 
> cycles spend on interrupt service, which is 40000 CPU cycles per packet.
> 

That depends on your application. It would be ok for normal desktop
PCs with fast CPU but it wouldn't be acceptable on servers that
have to do lots of other processing. If you have fxp(4) or txp(4)
hardwares give them try first and see what's the difference with
systat(1). Pushing the hardware to the limit by sending/receiving
64 bytes frames with netperf/iperf also would be good way to see
how well the controller works under extreme loads.
From babkin at verizon.net  Mon Jun 22 00:22:52 2009
From: babkin at verizon.net (Sergey Babkin)
Date: Mon Jun 22 02:17:00 2009
Subject: is RTL8139 THAT bad?
References: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
	<86eitdy4hl.fsf@ds4.des.no>
	<alpine.BSF.2.00.0906212219190.77226@wojtek.tensor.gdynia.pl>
Message-ID: <4A3EC37B.F589EA09@verizon.net>

Wojciech Puchar wrote:
> 
> > Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> writes:
> >> Why it's THAT bad?
> >
> > http://svn.freebsd.org/base/head/sys/pci/if_rl.c
> >
> > Scroll down past the copyright, license and attribution.  Read the
> > 38-line comment that explains just how crappy this chip really is.
> 
> Well - really "low end".
> 
> But - this computer can do memcpy at 80MB/s, so at 3.5MB/s it should be 5%
> CPU for memcpy, and one interrupt per one packet (2500 packets/s).
> 
> Is something more that make it consume >50% CPU?

Accessing the on-card memory through PCI is guaranteed to be
slower than the main memory, and depending on the particular
card it may be much slower.

-SB
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun 22 11:12:07 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun 22 11:12:14 2009
Subject: is RTL8139 THAT bad?
In-Reply-To: <4A3EC37B.F589EA09@verizon.net>
References: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
	<86eitdy4hl.fsf@ds4.des.no>
	<alpine.BSF.2.00.0906212219190.77226@wojtek.tensor.gdynia.pl>
	<4A3EC37B.F589EA09@verizon.net>
Message-ID: <alpine.BSF.2.00.0906221310140.28638@wojtek.tensor.gdynia.pl>

>>
>> But - this computer can do memcpy at 80MB/s, so at 3.5MB/s it should be 5%
>> CPU for memcpy, and one interrupt per one packet (2500 packets/s).
>>
>> Is something more that make it consume >50% CPU?
>
> Accessing the on-card memory through PCI is guaranteed to be
> slower than the main memory, and depending on the particular
> card it may be much slower.

as comment say - this card do DMA to main memory then computer must copy.

so PCI speed needs just to be faster than 100Mbit/s, certainly is
From wojtek at wojtek.tensor.gdynia.pl  Mon Jun 22 11:13:09 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Mon Jun 22 11:13:26 2009
Subject: is RTL8139 THAT bad?
In-Reply-To: <20090622001705.GA10712@michelle.cdnetworks.co.kr>
References: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
	<20090622001705.GA10712@michelle.cdnetworks.co.kr>
Message-ID: <alpine.BSF.2.00.0906221311210.28638@wojtek.tensor.gdynia.pl>

>>
>> Why it's THAT bad?
>>
>
> Because CPU always have to copy frames to/from the controller.

comment says card do DMA. just then it has to copy but within main memory 
not PCI.

> These CPU cycles could have been used in other task to give more
> performance such as SSH encryption/decryption, checksum computation
> etc.
>
>> 3.5MB/s is less that 2500 packets/second. 50% at 200Mhz means 100000000
>> cycles spend on interrupt service, which is 40000 CPU cycles per packet.
>>
>
> That depends on your application. It would be ok for normal desktop
> PCs with fast CPU but it wouldn't be acceptable on servers that
> have to do lots of other processing. If you have fxp(4) or txp(4)

i know all this, but i'm asking why processing single interrupt takes 
40000 CPU cycles.
From psteele at webmail.maxiscale.com  Mon Jun 22 18:42:49 2009
From: psteele at webmail.maxiscale.com (Peter Steele)
Date: Mon Jun 22 18:53:16 2009
Subject: Number of open files per process
Message-ID: <B8A480488C0C6849826655761349EA43128ADB@owa.webmail.maxiscale.com>

Is it possible to determine the number of open files per process? We
want to monitor this via a separate process and issue an alarm if some
threshold is crossed.

 

From wmoran at collaborativefusion.com  Mon Jun 22 19:07:24 2009
From: wmoran at collaborativefusion.com (Bill Moran)
Date: Mon Jun 22 19:07:31 2009
Subject: Number of open files per process
In-Reply-To: <B8A480488C0C6849826655761349EA43128ADB@owa.webmail.maxiscale.com>
References: <B8A480488C0C6849826655761349EA43128ADB@owa.webmail.maxiscale.com>
Message-ID: <20090622145720.768277c0.wmoran@collaborativefusion.com>

In response to "Peter Steele" <psteele@webmail.maxiscale.com>:

> Is it possible to determine the number of open files per process? We
> want to monitor this via a separate process and issue an alarm if some
> threshold is crossed.

The fstat command will do this for you.

-- 
Bill Moran
Collaborative Fusion Inc.
http://people.collaborativefusion.com/~wmoran/

wmoran@collaborativefusion.com
Phone: 412-422-3463x4023

****************************************************************
IMPORTANT: This message contains confidential information and is
intended only for the individual named. If the reader of this
message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.
****************************************************************
From shuvaev at physik.uni-wuerzburg.de  Mon Jun 22 19:36:16 2009
From: shuvaev at physik.uni-wuerzburg.de (Alexey Shuvaev)
Date: Mon Jun 22 19:36:23 2009
Subject: Number of open files per process
In-Reply-To: <B8A480488C0C6849826655761349EA43128ADB@owa.webmail.maxiscale.com>
References: <B8A480488C0C6849826655761349EA43128ADB@owa.webmail.maxiscale.com>
Message-ID: <20090622190529.GA52783@wep4035.physik.uni-wuerzburg.de>

On Mon, Jun 22, 2009 at 11:12:34AM -0700, Peter Steele wrote:
> Is it possible to determine the number of open files per process? We
> want to monitor this via a separate process and issue an alarm if some
> threshold is crossed.
> 
procstat -f <pid>
From xorquewasp at googlemail.com  Tue Jun 23 00:31:23 2009
From: xorquewasp at googlemail.com (xorquewasp@googlemail.com)
Date: Tue Jun 23 00:31:56 2009
Subject: ZFS filesystem not showing total size?
Message-ID: <20090623003117.GA94466@logik.internal.network>

Hello.

I've got the following hardware setup:

ad10: 76319MB <WDC WD800AAJS-00WAA0 58.01D58> at ata5-master SATA300
ad11: 953869MB <WDC WD1001FALS-00J7B1 05.00K05> at ata5-slave SATA300
ad12: 953869MB <WDC WD10EADS-00L5B1 01.01A01> at ata6-master SATA300
ad14: 476940MB <WDC WD5001AALS-00L3B2 01.03B01> at ata7-master SATA300
ad16: 476940MB <WDC WD5001AALS-00L3B2 01.03B01> at ata8-master SATA300

ad12 is used as a single-disk pool. ad10 is formatted UFS2 for the OS.

I'm attempting to create a RAIDZ pool (with parity) over ad11, ad14
and ad16.

I've performed the following steps:

# zpool create storage raidz ad11 ad14 ad16
# zpool list
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
backup                  928G    166K    928G     0%  ONLINE     -
storage                1.36T    292K   1.36T     0%  ONLINE     -

# zfs create storage/home
# zfs create storage/home/xw
# zfs set compression=gzip storage/home
# zfs set compression=gzip storage/home/xw

# zfs list
NAME              USED  AVAIL  REFER  MOUNTPOINT
storage           188K   913G  25.3K  /storage
storage/home     49.3K   913G  25.3K  /storage/home
storage/home/xw  24.0K   913G  24.0K  /storage/home/xw

I seem to have less storage than I'd expect. Is there something wrong
with this setup?

xw
From jflowers at ezo.net  Tue Jun 23 04:52:47 2009
From: jflowers at ezo.net (Jim Flowers)
Date: Tue Jun 23 04:53:33 2009
Subject: dd copy of FreeBSD-7.2 won't boot
Message-ID: <20090623032742.M54551@ezo.net>

I have a remote server that was dd copied from one hard drive to another - 
essentially the same size.  The disk device name (ad4) is the same but the 
geometry for the new drive has a CHS of 969021/16/63

On booting it hangs at:

F1        FreeBSD
Boot:     F1

I copied the MBR with 'boot0cfg -B -opacket ad4' just to be sure but no joy.  
fbsd fdisk reports start 63, with CHS beg: 0/1/1 end: 1023/15/63.

Any help on direction to solve this?

Thanks.

--
Jim Flowers <jflowers@ezo.net>

From fbsdlist at src.cx  Tue Jun 23 04:55:06 2009
From: fbsdlist at src.cx (Artem Belevich)
Date: Tue Jun 23 04:55:13 2009
Subject: ZFS filesystem not showing total size?
In-Reply-To: <20090623003117.GA94466@logik.internal.network>
References: <20090623003117.GA94466@logik.internal.network>
Message-ID: <ed91d4a80906222128y6098bdafk67d67758a36691ec@mail.gmail.com>

On Mon, Jun 22, 2009 at 5:31 PM, <xorquewasp@googlemail.com> wrote:
> ad11: 953869MB <WDC WD1001FALS-00J7B1 05.00K05> at ata5-slave SATA300
..
> ad14: 476940MB <WDC WD5001AALS-00L3B2 01.03B01> at ata7-master SATA300
> ad16: 476940MB <WDC WD5001AALS-00L3B2 01.03B01> at ata8-master SATA300
..
># zpool create storage raidz ad11 ad14 ad16
> I seem to have less storage than I'd expect. Is there something wrong
> with this setup?

You've created raidz pool from 2x500G hard drives and one 1T hard
drive. RAIDZ needs elements of the array to be of the same size, so
you're effectively wasting half of your 1T drive (ad11). Total raw
capacity of the pool is 476G*3 = ~ 1.36T with 2/3of it (913G) usable
for user data -- exactly what zfs shows you.

Obvious fix would be to build the pool from the drives of the same
size. You may try concatenating or striping ad14+ad16 with geom and
create mirrored pool with ad11 -- usable size would be a bit larger
than what you have. Resulting pool should be faster than RAIDZ.

Another option would be to get another 1T drive and build RAIDZ pool
with 2x1T + (striped 2x500G) for a total usable size of ~2T.

I'd advise against splitting 1T drive into two partitions and use them
as two elements of RAIDZ pool -- this would effectively kill
redundancy that RAIDZ is supposed to provide. If your 1T drive dies,
your whole pool would go down with it. Performance would also suck as
1T drive will be constantly seeking.

--Artem
From xorquewasp at googlemail.com  Tue Jun 23 05:07:11 2009
From: xorquewasp at googlemail.com (xorquewasp@googlemail.com)
Date: Tue Jun 23 05:07:18 2009
Subject: ZFS filesystem not showing total size?
In-Reply-To: <ed91d4a80906222128y6098bdafk67d67758a36691ec@mail.gmail.com>
References: <20090623003117.GA94466@logik.internal.network>
	<ed91d4a80906222128y6098bdafk67d67758a36691ec@mail.gmail.com>
Message-ID: <20090623050707.GB21349@logik.internal.network>

On 2009-06-22 21:28:26, Artem Belevich wrote:
> On Mon, Jun 22, 2009 at 5:31 PM, <xorquewasp@googlemail.com> wrote:
> > ad11: 953869MB <WDC WD1001FALS-00J7B1 05.00K05> at ata5-slave SATA300
> ..
> > ad14: 476940MB <WDC WD5001AALS-00L3B2 01.03B01> at ata7-master SATA300
> > ad16: 476940MB <WDC WD5001AALS-00L3B2 01.03B01> at ata8-master SATA300
> ..
> ># zpool create storage raidz ad11 ad14 ad16
> > I seem to have less storage than I'd expect. Is there something wrong
> > with this setup?
> 
> You've created raidz pool from 2x500G hard drives and one 1T hard
> drive. RAIDZ needs elements of the array to be of the same size, so
> you're effectively wasting half of your 1T drive (ad11). Total raw
> capacity of the pool is 476G*3 = ~ 1.36T with 2/3of it (913G) usable
> for user data -- exactly what zfs shows you.

Thanks, I was afraid of that.

I'll swap the 1tb disk for a 500gb today. I was only planning to get
1tb of storage but found that I had a spare 1tb disk laying around so
I put that into the array instead.

xw
From doconnor at gsoft.com.au  Tue Jun 23 06:39:29 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Tue Jun 23 06:39:39 2009
Subject: dd copy of FreeBSD-7.2 won't boot
In-Reply-To: <20090623032742.M54551@ezo.net>
References: <20090623032742.M54551@ezo.net>
Message-ID: <200906231609.05810.doconnor@gsoft.com.au>

On Tue, 23 Jun 2009, Jim Flowers wrote:
> I have a remote server that was dd copied from one hard drive to
> another - essentially the same size.  The disk device name (ad4) is
> the same but the geometry for the new drive has a CHS of 969021/16/63
>
> On booting it hangs at:
>
> F1        FreeBSD
> Boot:     F1
>
> I copied the MBR with 'boot0cfg -B -opacket ad4' just to be sure but
> no joy. fbsd fdisk reports start 63, with CHS beg: 0/1/1 end:
> 1023/15/63.
>
> Any help on direction to solve this?

I would have thought that boot0cfg would DTRT.

You could also try running fdisk -BI on your new disk to reinit the MBR, 
then running boot0cfg on it.

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090623/f881c73b/attachment.pgp
From wojtek at wojtek.tensor.gdynia.pl  Tue Jun 23 07:29:09 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Tue Jun 23 07:29:16 2009
Subject: dd copy of FreeBSD-7.2 won't boot
In-Reply-To: <20090623032742.M54551@ezo.net>
References: <20090623032742.M54551@ezo.net>
Message-ID: <alpine.BSF.2.00.0906230927390.55064@wojtek.tensor.gdynia.pl>

> I have a remote server that was dd copied from one hard drive to another -
> essentially the same size.  The disk device name (ad4) is the same but the
> geometry for the new drive has a CHS of 969021/16/63
>
> On booting it hangs at:
>
> F1        FreeBSD
> Boot:     F1
>
> I copied the MBR with 'boot0cfg -B -opacket ad4' just to be sure but no joy.
> fbsd fdisk reports start 63, with CHS beg: 0/1/1 end: 1023/15/63.
>
> Any help on direction to solve this?
>

try fdisk -B /dev/ad0

but even better next time don't make slices, only disklabel. It just make 
life simpler. I still don't understand why sysinstall by default create 
them. It should be only used when windoze has to be run from the same disk
From doconnor at gsoft.com.au  Tue Jun 23 07:34:11 2009
From: doconnor at gsoft.com.au (Daniel O'Connor)
Date: Tue Jun 23 07:34:18 2009
Subject: dd copy of FreeBSD-7.2 won't boot
In-Reply-To: <alpine.BSF.2.00.0906230927390.55064@wojtek.tensor.gdynia.pl>
References: <20090623032742.M54551@ezo.net>
	<alpine.BSF.2.00.0906230927390.55064@wojtek.tensor.gdynia.pl>
Message-ID: <200906231703.58948.doconnor@gsoft.com.au>

On Tue, 23 Jun 2009, Wojciech Puchar wrote:
> > Any help on direction to solve this?
>
> try fdisk -B /dev/ad0
>
> but even better next time don't make slices, only disklabel. It just
> make life simpler. I still don't understand why sysinstall by default
> create them. It should be only used when windoze has to be run from
> the same disk

There are plenty of BIOSen which will puke on a dangerously dedicated 
disk.

Next time he should partition, label & newfs the disk, then copy it over 
with dump | store.

Unless the disk is chock full it will be significantly faster.
 

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: This is a digitally signed message part.
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090623/31090a4c/attachment.pgp
From peterjeremy at optushome.com.au  Tue Jun 23 07:37:00 2009
From: peterjeremy at optushome.com.au (peterjeremy@optushome.com.au)
Date: Tue Jun 23 07:37:07 2009
Subject: is RTL8139 THAT bad?
In-Reply-To: <alpine.BSF.2.00.0906221311210.28638@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906211709420.1184@wojtek.tensor.gdynia.pl>
	<20090622001705.GA10712@michelle.cdnetworks.co.kr>
	<alpine.BSF.2.00.0906221311210.28638@wojtek.tensor.gdynia.pl>
Message-ID: <20090623073647.GA7501@server.vk2pj.dyndns.org>

On 2009-Jun-22 13:12:08 +0200, Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> wrote:
>i know all this, but i'm asking why processing single interrupt takes 
>40000 CPU cycles.

All I can suggest is that you browse the sources and see what the
rl(4) interrupt handler does.  If you want to dig further, hwpmc(4)
may give you further insights.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090623/304b26db/attachment.pgp
From eforezz at gmail.com  Tue Jun 23 14:24:23 2009
From: eforezz at gmail.com (EforeZZ)
Date: Tue Jun 23 14:24:31 2009
Subject: is RTL8139 THAT bad?
Message-ID: <b09816520906230703p40299553kcc1668e22c236de8@mail.gmail.com>

>i have pentium 200 with that card. doing ftp from other machine, getting
>3.5MB/s (HDD can 10MB/s, DMA) having 45%-55% interrupt load.

I had the same probelm. I rebuilt the kernel with the POLLING option
set to ON and turned on polling for all network interfaces. It helped
much :)

Best regards,
EforeZZ
From jgreco at ns.sol.net  Tue Jun 23 18:09:19 2009
From: jgreco at ns.sol.net (Joe Greco)
Date: Tue Jun 23 18:09:25 2009
Subject: Maybe confused about AMD64 / i386 compatibility
In-Reply-To: <20090613230749.GA73896@server.vk2pj.dyndns.org> from "Peter
	Jeremy" at Jun 14, 2009 09:07:50 AM
Message-ID: <200906231809.n5NI9Nj0002648@aurora.sol.net>

> On 2009-Jun-13 15:55:29 -0500, Joe Greco <jgreco@ns.sol.net> wrote:
> >Adding a SIL3112A gives us the SATA.
> 
> These are known to cause data corruption (check the archives).  I
> wouldn't trust anything that has passed through a SIL chip without
> independent validation.

Well, as I originally stated, that wasn't an issue.

The problem has been replicated on other hardware and has been assigned
to freebsd-geom, I guess.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
From thacker.nirmal at gmail.com  Tue Jun 23 20:24:28 2009
From: thacker.nirmal at gmail.com (Nirmal Thacker)
Date: Tue Jun 23 20:24:40 2009
Subject: Dump Utility cache efficiency analysis
Message-ID: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>

Hello

This is regarding the dump utility cache efficiency analysis post made on
February '07 by Peter Jeremy [
http://lists.freebsd.org/pipermail/freebsd-hackers/2007-February/019666.html]
and if this project is still open. I would be interested to begin exploring
FreeBSD (and contributing) by starting this project.

I do have some basic understanding of the problem at hand - to determine if
a unified cache would appeal as a more efficient/elegant solution compared
to the per-process-cache in the Dump utility implementation. I admit I am
new to this list and FreeBSD so I wouldn't be able to determine what the
current implementation is, until I get started.

I would first like to understand the opinions of anyone who has looked at
this problem or think this would be a worthwhile project to start off with.

I would also appreciate if I could get simple tips and pointers of setting
up my machine for the project. I understand this would be on the lines of:

1. Installing a stable FreeBSD build
2. Check out a version of the Build suitable for the project
3. Pointers to begin studying the current implementation in the code-tree
structure (would I expect it to lie in the fs/ directory?). I tried to find
it in the FreeBSD cross reference (http://fxr.watson.org/)
4. Read some important sections of the developer handbook (some suggestions
would be great)

Lastly- does this project require the know-how's of device drivers? If so, I
would have to work harder.

Thanks a lot!

- nirmal
From freebsd at akruijff.dds.nl  Wed Jun 24 00:08:33 2009
From: freebsd at akruijff.dds.nl (Alex de Kruijff)
Date: Wed Jun 24 00:08:40 2009
Subject: [FIXED] Re: Porting problem with gnu configure (c++ -V)
In-Reply-To: <20090613110618.GA1462@Alex1.lan>
References: <20090613110618.GA1462@Alex1.lan>
Message-ID: <20090624000829.GB1697@Alex1.lan>

On Sat, Jun 13, 2009 at 01:06:18PM +0200, Alex de Kruijff wrote:
> I'm converting my port samesame to use gnu configue, but came a cross a
> problem that is beond me. I'm able to run aclocal, autoconf, autoheader
> and automake --add-missing -c. I've read the docs for autoconf and
> automake and search the web, but dont know how to solve elegantly. I
> would proberbly be able to hack configure, but prevere to edit only the
> source files. Do other porters with more experiance have any tips for
> me?
> 
> The machine runs on FreeBSD 6.1-p20.
> 
> When building the port I get this message:
> 
> ===>  Configuring for samesame-1.3
> configure: WARNING: you should use --build, --host, --target
> checking for a BSD-compatible install... /usr/bin/install -c -o root -g
> wheel
> checking whether build environment is sane... yes
> checking for gawk... no
> checking for mawk... no
> checking for nawk... nawk
> checking whether make sets $(MAKE)... yes
> checking for C++ compiler default output file name...
> configure: error: C++ compiler cannot create executables
> See `config.log' for more details.
> ===>  Script "configure" failed unexpectedly.
> 
> 
> - Config.log show the following:
> configure:2329: c++ --version >&5
> c++ (GCC) 3.4.4 [FreeBSD] 20050518
> Copyright (C) 2004 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is
> NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.
> 
> configure:2332: $? = 0
> configure:2339: c++ -v >&5
> Using built-in specs.   
> Configured with: FreeBSD/i386 system compiler
> Thread model: posix
> gcc version 3.4.4 [FreeBSD] 20050518
> configure:2342: $? = 0
> configure:2349: c++ -V >&5 <---------------------------- ------- -------
> c++: `-V' option must have argument
> configure:2352: $? = 1
> configure:2375: checking for C++ compiler default output file name
> configure:2402: c++ -O2 -fno-strict-aliasing -pipe -DWITH_DISK_STORAGE
> -DWITH_MM
> AP -D'TEMP_STORAGE_DIR="/tmp"' -DPATH_INIT=256
> -DSTATIC_CACHE_CAPACITY=8192   co
> nftest.cpp  >&5
> <command line>:4:1: macro names must be identifiers
> configure:2405: $? = 1
> configure:2443: result:
> configure: failed program was:
> | /* confdefs.h.  */ /
> | #define PACKAGE_NAME "SameSame"
> | #define PACKAGE_TARNAME "samesame"
> | #define PACKAGE_VERSION "1.3"
> | #define PACKAGE_STRING "SameSame 1.3"
> | #define PACKAGE_BUGREPORT "samesame@akruijff.dds.nl"
> | #define PACKAGE "samesame"
> | #define VERSION "1.3"
> | /* end confdefs.h.  */
> |
> | int
> | main ()
> | {
> |
> |   ;
> |   return 0;
> | }
> configure:2449: error: C++ compiler cannot create executables
> See `config.log' for more details.
> 

Hi,

For the archive (google). I that I discoverd the problem with the help
of Andrey Simonenko. The problem with the port seed to be that you can
not do stuff like 'CFLAGS+= -DWITH_MMAP' when going the gnu way with
your port. This causes the strange error messages. Removing this will
allow the configure script to be called correctly.

-- 
Alex

http://samesame.kruijff.org/

From dillon at apollo.backplane.com  Wed Jun 24 01:07:53 2009
From: dillon at apollo.backplane.com (Matthew Dillon)
Date: Wed Jun 24 01:07:59 2009
Subject: Dump Utility cache efficiency analysis
References: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>
Message-ID: <200906240107.n5O17q9j015112@apollo.backplane.com>

:Hello
:
:This is regarding the dump utility cache efficiency analysis post made on
:February '07 by Peter Jeremy [
:http://lists.freebsd.org/pipermail/freebsd-hackers/2007-February/019666.html]
:and if this project is still open. I would be interested to begin exploring
:FreeBSD (and contributing) by starting this project.
:
:I do have some basic understanding of the problem at hand - to determine if
:a unified cache would appeal as a more efficient/elegant solution compared
:to the per-process-cache in the Dump utility implementation. I admit I am
:new to this list and FreeBSD so I wouldn't be able to determine what the
:current implementation is, until I get started.
:...

     I think the cache in the dump utility is still the one I worked up
     a long time ago.  It was a quick and dirty job at the time, and it
     was never really designed for parallel operation which is probably
     why it doesn't work so well in that regard.

     In my opinion, a unified cache would be an excellent improvement.
     Ultimately dump is an I/O bound process so I don't think we would
     really need to worry about the minor increases in cpu overhead
     from the additional locking needed.

     There are a few issues you will have to consider:

     * Dump uses a fork model for its children rather then pthreads.  You
       would either have to use the F_*LK fcntl() operations or use a
       simpler flock() scheme to lock across the children.  Alternatively
       you could change dump over to a pthreads model and use pthreads
       mutexes, but that would entail a lot more work.  Dump was never
       designed to be threaded.

     * The general issue with any caching scheme for dump is how much to
       actually cache per I/O vs the size of the cache.  Caching larger
       amounts of data hits diminishing returns as it also increases seek
       times and waste (cached data never usde).  Caching smaller amounts
       of data hits diminishing returns as it causes the disk to seek more.

    Disk drives generally do have a track cache, but they also only typically
    have 8-16M of cache ram (32M in newer drives, particularly the higher
    capacity ones).  A track is typically about 1-2M (maybe higher now) so
    it doesn't take much seeking for the drive to blow out its internal
    track cache.  Caching that much data in a single read would probably
    be detrimental anyway.

    This also means you do not necessarily want to cache too much
    linearly-read data, as the disk drive is already doing it for you.

    Because of all of this it is going to be tough to find cache parameters
    that work well generally, and the parameters are going to chance
    drastically based on the amount of cache you specify on the command
    line and the size of the partition being dumped.

						-Matt


From xorquewasp at googlemail.com  Wed Jun 24 01:48:59 2009
From: xorquewasp at googlemail.com (xorquewasp@googlemail.com)
Date: Wed Jun 24 01:49:06 2009
Subject: ZFS filesystem not showing total size?
In-Reply-To: <20090624004726.GA25475@blazingdot.com>
References: <20090623003117.GA94466@logik.internal.network>
	<ed91d4a80906222128y6098bdafk67d67758a36691ec@mail.gmail.com>
	<20090623050707.GB21349@logik.internal.network>
	<20090624004726.GA25475@blazingdot.com>
Message-ID: <20090624014855.GB79749@logik.internal.network>

On 2009-06-23 17:47:26, Marcus Reid wrote:
> Hi,
> 
> As a side note, freebsd-hackers is not the correct list for this question.

I considered it a hackers@ question due to the large "ZFS is an experimental
feature" warning.

xw
From danny at cs.huji.ac.il  Wed Jun 24 07:52:05 2009
From: danny at cs.huji.ac.il (Danny Braniss)
Date: Wed Jun 24 07:52:13 2009
Subject: Dump Utility cache efficiency analysis 
In-Reply-To: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com> 
References: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>
Message-ID: <E1MJNH1-000HXd-6E@kabab.cs.huji.ac.il>

> Hello
> 
> This is regarding the dump utility cache efficiency analysis post made on
> February '07 by Peter Jeremy [
> http://lists.freebsd.org/pipermail/freebsd-hackers/2007-February/019666.html]
> and if this project is still open. I would be interested to begin exploring
> FreeBSD (and contributing) by starting this project.
> 
> I do have some basic understanding of the problem at hand - to determine if
> a unified cache would appeal as a more efficient/elegant solution compared
> to the per-process-cache in the Dump utility implementation. I admit I am
> new to this list and FreeBSD so I wouldn't be able to determine what the
> current implementation is, until I get started.
> 
> I would first like to understand the opinions of anyone who has looked at
> this problem or think this would be a worthwhile project to start off with.
> 
> I would also appreciate if I could get simple tips and pointers of setting
> up my machine for the project. I understand this would be on the lines of:
> 
> 1. Installing a stable FreeBSD build
> 2. Check out a version of the Build suitable for the project
> 3. Pointers to begin studying the current implementation in the code-tree
> structure (would I expect it to lie in the fs/ directory?). I tried to find
> it in the FreeBSD cross reference (http://fxr.watson.org/)
> 4. Read some important sections of the developer handbook (some suggestions
> would be great)
> 
> Lastly- does this project require the know-how's of device drivers? If so, I
> would have to work harder.
> 
short answer:
	you don't need driver knowledge, but fs is a must.
long answer:
	In the days long gone, the cpu/disk where slower than the tape,
which could 'stream', and unless you could provide data fast enough, the tape
would stop, rewind some, then pick up speed, and write.
	Nowadays, tapes are slower, but some/most of us dump to file, or
pipe to restore (dump -f - ... | restore rf -), so that the tape speed is
irrelevant. On the other hand, computers have much more memory, so buffering
can be done by the OS.
	What I'm trying to say, and not wanting to take out any air from
from the sails, is that dump should be re-valuated, and maybe OpenBSD/KIS
is the best.

	danny



> Thanks a lot!
> 
> - nirmal
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
> 


From peterjeremy at optushome.com.au  Wed Jun 24 07:58:16 2009
From: peterjeremy at optushome.com.au (Peter Jeremy)
Date: Wed Jun 24 07:58:23 2009
Subject: Dump Utility cache efficiency analysis
In-Reply-To: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>
References: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>
Message-ID: <20090624075811.GA463@server.vk2pj.dyndns.org>

On 2009-Jun-23 15:52:04 -0400, Nirmal Thacker <thacker.nirmal@gmail.com> wrote:
>I would first like to understand the opinions of anyone who has looked at
>this problem or think this would be a worthwhile project to start off with.

I'm aware of the following references:
http://www.mavetju.org/mail/view_message.php?list=freebsd-hackers&id=375676
http://www.mavetju.org/mail/view_thread.php?list=freebsd-stable&id=1335519&thread=yes

>1. Installing a stable FreeBSD build
>2. Check out a version of the Build suitable for the project

Any changes will need to apply to FreeBSD -current, though they may be
back-ported once tested.  This means that you will need a -current
system at some point.  8-current is reasonably stable at this point and
would be my suggestion.

>3. Pointers to begin studying the current implementation in the code-tree
>structure (would I expect it to lie in the fs/ directory?). I tried to find
>it in the FreeBSD cross reference (http://fxr.watson.org/)

The code is in src/sbin/dump.  It references various system header
files in order to understand the UFS on-disk format.

>Lastly- does this project require the know-how's of device drivers? If so, I
>would have to work harder.

No.  Dump is completely userland.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090624/9e9a338c/attachment.pgp
From alexbestms at math.uni-muenster.de  Wed Jun 24 09:41:53 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Wed Jun 24 09:42:00 2009
Subject: small usr.bin/find patch
In-Reply-To: <alpine.BSF.2.00.0906172325310.36849@wojtek.tensor.gdynia.pl>
Message-ID: <permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>

hmmm...but dd e.g. uses lowercase instead of upercase letters to indicate
kilobyte, megabyte and so on. isn't there some unix/posix/whatever standard
telling app developers what to use?

Wojciech Puchar schrieb am 2009-06-17:

> >>cheers.


> >Are you sure this is wise? after all 125 millibytes would be 1 bit..

> Agree. While lots of people use m instead of M and b instead of B,
> this is not right to correct proper behavior to improper just for
> them.
From m.seaman at infracaninophile.co.uk  Wed Jun 24 10:31:41 2009
From: m.seaman at infracaninophile.co.uk (Matthew Seaman)
Date: Wed Jun 24 10:31:48 2009
Subject: small usr.bin/find patch
In-Reply-To: <permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
References: <permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
Message-ID: <4A420073.6060405@infracaninophile.co.uk>

Alexander Best wrote:
> hmmm...but dd e.g. uses lowercase instead of upercase letters to indicate
> kilobyte, megabyte and so on. isn't there some unix/posix/whatever standard
> telling app developers what to use?

Sure. The standard for scale-prefixes is defined by the Systeme
Internationale as part of the definition of SI units:

  http://www.npl.co.uk/reference/measurement-units/si-prefixes/

Note that these are strictly powers-of-10^3 multipliers, and explicitly
not the computing style powers-of-2^10 commonly used for file sizes or
hard drive capacities, which should instead use the somewhat clunky Ki,
Mi, Gi etc. forms:

  http://physics.nist.gov/cuu/Units/binary.html

These binary prefixes are mandated by the IEC and approved by the IEEE
amongst others.

Not that many people use the binary prefixes appropriately, relying on
context to disambiguate 1 MB = 1024 KB = 1,048,576 Bytes etc.  Except
that (confusingly) as a measure of network bandwidth 10 Mb/s always was
10,000,000 b/s and never 10,485,760 b/s; a fact that has caught me out
more than a few times.

Making find(1) / dd(1) / etc. operate pedantically correctly with these
scale-factor symbols would cause a certain degree of pain for little
practical gain.  Unless there was a broad consensus amongst all Unixoid
OS providers, I can't see that change ever happening.

	Cheers,

	Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.                       Flat 3
                                                      7 Priory Courtyard
PGP: http://www.infracaninophile.co.uk/pgpkey         Ramsgate
                                                      Kent, CT11 9PW, UK

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 259 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090624/d8ff626a/signature.pgp
From johans at stack.nl  Wed Jun 24 10:33:29 2009
From: johans at stack.nl (Johan van Selst)
Date: Wed Jun 24 10:33:35 2009
Subject: small usr.bin/find patch
In-Reply-To: <permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
References: <alpine.BSF.2.00.0906172325310.36849@wojtek.tensor.gdynia.pl>
	<permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
Message-ID: <20090624095000.GA85397@mud.stack.nl>

Alexander Best wrote:
> hmmm...but dd e.g. uses lowercase instead of upercase letters to indicate
> kilobyte, megabyte and so on. isn't there some unix/posix/whatever standard
> telling app developers what to use?

It might be appropriate to use expand_number() here. This is what some
other tools do as well and consistency between tools is "a nice thing".


Ciao,
Johan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090624/93403f63/attachment.pgp
From Alexander at Leidinger.net  Wed Jun 24 06:05:58 2009
From: Alexander at Leidinger.net (Alexander Leidinger)
Date: Wed Jun 24 13:13:35 2009
Subject: Dump Utility cache efficiency analysis
In-Reply-To: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>
References: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>
Message-ID: <20090624074834.00005bd9@unknown>

On Tue, 23 Jun 2009 15:52:04 -0400 Nirmal Thacker
<thacker.nirmal@gmail.com> wrote:

> I would also appreciate if I could get simple tips and pointers of
> setting up my machine for the project. I understand this would be on
> the lines of:
> 
> 1. Installing a stable FreeBSD build
> 2. Check out a version of the Build suitable for the project

All development is taking place in -CURRENT, so you would have to check
out this one, or you install it right away, it's not declared stable
yet, but as we have started the release management process for 8.0,
it's not that unstable either... :)

> 3. Pointers to begin studying the current implementation in the
> code-tree structure (would I expect it to lie in the fs/ directory?).
> I tried to find it in the FreeBSD cross reference
> (http://fxr.watson.org/) 4. Read some important sections of the
> developer handbook (some suggestions would be great)

Dump is a complete userland implementation. All you need to know is the
userland programming stuff, specially for what you want to do. You can
find it online at
   http://svnweb.freebsd.org/viewvc/base/head/sbin/dump/

Bye,
Alexander.
From johans at stack.nl  Wed Jun 24 09:50:04 2009
From: johans at stack.nl (Johan van Selst)
Date: Wed Jun 24 13:13:57 2009
Subject: small usr.bin/find patch
In-Reply-To: <permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
References: <alpine.BSF.2.00.0906172325310.36849@wojtek.tensor.gdynia.pl>
	<permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
Message-ID: <20090624095000.GA85397@mud.stack.nl>

Alexander Best wrote:
> hmmm...but dd e.g. uses lowercase instead of upercase letters to indicate
> kilobyte, megabyte and so on. isn't there some unix/posix/whatever standard
> telling app developers what to use?

It might be appropriate to use expand_number() here. This is what some
other tools do as well and consistency between tools is "a nice thing".


Ciao,
Johan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090624/93403f63/attachment-0002.pgp
From lars.engels at 0x20.net  Wed Jun 24 12:17:02 2009
From: lars.engels at 0x20.net (Lars Engels)
Date: Wed Jun 24 13:14:05 2009
Subject: small usr.bin/find patch
In-Reply-To: <permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
References: <permail-20090624094151f0889e8400007f8d-a_best01@message-id.uni-muenster.de>
Message-ID: <20090624135717.j7wnshs43oog4gwo@0x20.net>

Quoting Alexander Best <alexbestms@math.uni-muenster.de>:

> hmmm...but dd e.g. uses lowercase instead of upercase letters to indicate
> kilobyte, megabyte and so on. isn't there some unix/posix/whatever standard
> telling app developers what to use?
>

Actually thats only BSD's dd. GNU dd only accepts uppercase letters.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: PGP Digital Signature
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090624/71c0ebac/attachment.pgp
From sean.bruno at dsl-only.net  Wed Jun 24 17:06:53 2009
From: sean.bruno at dsl-only.net (Sean Bruno)
Date: Wed Jun 24 17:07:25 2009
Subject: 8 week projects
Message-ID: <1245861612.3381.6.camel@localhost.localdomain>

My open source class this summer has a lot of people in it looking for 8
week projects.

If you have a decently spec'd out project that a Junior/Senior CS
student can accomplish, send me a link or pointer to it and I'll see if
I can get the project some attention.

Sean

From thacker.nirmal at gmail.com  Wed Jun 24 17:59:10 2009
From: thacker.nirmal at gmail.com (Nirmal Thacker)
Date: Wed Jun 24 17:59:17 2009
Subject: Dump Utility cache efficiency analysis
In-Reply-To: <20090624075811.GA463@server.vk2pj.dyndns.org>
References: <87429ffe0906231252j7c84489dt6ebd60333654f411@mail.gmail.com>
	<20090624075811.GA463@server.vk2pj.dyndns.org>
Message-ID: <87429ffe0906241059x5e2d083bo626d4ba0c9b1cd44@mail.gmail.com>

Thanks for all the replies and suggestions
I ll begin by running, benchmarking, understanding dump for myself and take
up Matt's suggestions above to understand the unified caching implementation
in more detail
-n

On Wed, Jun 24, 2009 at 3:58 AM, Peter Jeremy
<peterjeremy@optushome.com.au>wrote:

> On 2009-Jun-23 15:52:04 -0400, Nirmal Thacker <thacker.nirmal@gmail.com>
> wrote:
> >I would first like to understand the opinions of anyone who has looked at
> >this problem or think this would be a worthwhile project to start off
> with.
>
> I'm aware of the following references:
> http://www.mavetju.org/mail/view_message.php?list=freebsd-hackers&id=375676
>
> http://www.mavetju.org/mail/view_thread.php?list=freebsd-stable&id=1335519&thread=yes
>
> >1. Installing a stable FreeBSD build
> >2. Check out a version of the Build suitable for the project
>
> Any changes will need to apply to FreeBSD -current, though they may be
> back-ported once tested.  This means that you will need a -current
> system at some point.  8-current is reasonably stable at this point and
> would be my suggestion.
>
> >3. Pointers to begin studying the current implementation in the code-tree
> >structure (would I expect it to lie in the fs/ directory?). I tried to
> find
> >it in the FreeBSD cross reference (http://fxr.watson.org/)
>
> The code is in src/sbin/dump.  It references various system header
> files in order to understand the UFS on-disk format.
>
> >Lastly- does this project require the know-how's of device drivers? If so,
> I
> >would have to work harder.
>
> No.  Dump is completely userland.
>
> --
> Peter Jeremy
>
From kraduk at googlemail.com  Wed Jun 24 22:11:26 2009
From: kraduk at googlemail.com (krad)
Date: Wed Jun 24 23:39:22 2009
Subject: dd copy of FreeBSD-7.2 won't boot
In-Reply-To: <20090623032742.M54551@ezo.net>
References: <20090623032742.M54551@ezo.net>
Message-ID: <2C153FCDAC3B43BDADDD1B28CED80036@uk.tiscali.intl>

Personally id stay away from dd. Create the partitions and file systems
manually, and install the boot loader, then rsync the data across. It will
be a lot faster in most cases, as unlike dd you wont be copying unused
space. Something like this should do the job

Rsync -aPH --exclude=/mnt/** / /mnt

I'm assuming you weren't migrating due to a bad disk

-----Original Message-----
From: owner-freebsd-hackers@freebsd.org
[mailto:owner-freebsd-hackers@freebsd.org] On Behalf Of Jim Flowers
Sent: 23 June 2009 05:55
To: freebsd-hackers@freebsd.org
Subject: dd copy of FreeBSD-7.2 won't boot

I have a remote server that was dd copied from one hard drive to another - 
essentially the same size.  The disk device name (ad4) is the same but the 
geometry for the new drive has a CHS of 969021/16/63

On booting it hangs at:

F1        FreeBSD
Boot:     F1

I copied the MBR with 'boot0cfg -B -opacket ad4' just to be sure but no joy.

fbsd fdisk reports start 63, with CHS beg: 0/1/1 end: 1023/15/63.

Any help on direction to solve this?

Thanks.

--
Jim Flowers <jflowers@ezo.net>

_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

From vince at unsane.co.uk  Thu Jun 25 09:19:22 2009
From: vince at unsane.co.uk (Vincent Hoffman)
Date: Thu Jun 25 09:19:29 2009
Subject: 8 week projects
In-Reply-To: <1245861612.3381.6.camel@localhost.localdomain>
References: <1245861612.3381.6.camel@localhost.localdomain>
Message-ID: <4A434109.3000107@unsane.co.uk>

On 24/6/09 17:40, Sean Bruno wrote:
> My open source class this summer has a lot of people in it looking for 8
> week projects.
>
> If you have a decently spec'd out project that a Junior/Senior CS
> student can accomplish, send me a link or pointer to it and I'll see if
> I can get the project some attention.
>
>   
Dont know about decently spec'd but
http://www.freebsd.org/projects/ideas/ has a bunch of ideas if thats any
help.

Vince
> Sean
>
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
>   

From alexbestms at math.uni-muenster.de  Thu Jun 25 12:15:52 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Thu Jun 25 12:16:00 2009
Subject: small usr.bin/find patch
In-Reply-To: <4A420073.6060405@infracaninophile.co.uk>
Message-ID: <permail-200906251215491e86ffa800006660-a_best01@message-id.uni-muenster.de>

wow. thanks a bunch for all the great info.

Matthew Seaman schrieb am 2009-06-24:
> Alexander Best wrote:
> > hmmm...but dd e.g. uses lowercase instead of upercase letters to
> > indicate
> > kilobyte, megabyte and so on. isn't there some unix/posix/whatever
> > standard
> > telling app developers what to use?

> Sure. The standard for scale-prefixes is defined by the Systeme
> Internationale as part of the definition of SI units:

>   http://www.npl.co.uk/reference/measurement-units/si-prefixes/

> Note that these are strictly powers-of-10^3 multipliers, and
> explicitly
> not the computing style powers-of-2^10 commonly used for file sizes
> or
> hard drive capacities, which should instead use the somewhat clunky
> Ki,
> Mi, Gi etc. forms:

>   http://physics.nist.gov/cuu/Units/binary.html

> These binary prefixes are mandated by the IEC and approved by the
> IEEE
> amongst others.

> Not that many people use the binary prefixes appropriately, relying
> on
> context to disambiguate 1 MB = 1024 KB = 1,048,576 Bytes etc.  Except
> that (confusingly) as a measure of network bandwidth 10 Mb/s always
> was
> 10,000,000 b/s and never 10,485,760 b/s; a fact that has caught me
> out
> more than a few times.

> Making find(1) / dd(1) / etc. operate pedantically correctly with
> these
> scale-factor symbols would cause a certain degree of pain for little
> practical gain.  Unless there was a broad consensus amongst all
> Unixoid
> OS providers, I can't see that change ever happening.

>         Cheers,

>         Matthew

From James.McPherson at Sun.COM  Thu Jun 25 05:32:20 2009
From: James.McPherson at Sun.COM (James C. McPherson)
Date: Thu Jun 25 14:05:08 2009
Subject: Another reason to register and attend Kernel Conference Australia
Message-ID: <20090625153151.00002b15@blinder>


Hello everybody,
If you were still in need of a reason to come to Kernel Conference
Australia, how about this:

Brendan Gregg, author of the DTraceToolkit and member of the
Fishworks team at Sun Microsystems

is going to be presenting on DTrace at Kernel Conference Australia.


Here's a short bio entry:

-------------------------
Brendan Gregg is a staff engineer in the Fishworks group at Sun Microsystems
where he works on a wide range of kernel and user-land technologies,
especially those related to performance. He is the creator of the
DTraceToolkit and is the co-author of "Solaris Performance and Tools".
His recent projects include writing the DTrace IP provider, the Fishworks
BUI dashboard, and the ZFS L2ARC.

Brendan has previously worked as a system administrator, Sun instructor
and consultant in the Asia-Pacific region, and was based in Sydney, Australia.
He is currently living in San Francisco, California, where he misses cricket
and rugby on TV.


http://blogs.sun.com/brendan
http://blogs.sun.com/fishworks
-------------------------

So don't delay any longer, seats are limited and running out.


It's not just about Sun's technologies, it's about Open Source.

Remember - innovation happens _everywhere_ (and DTrace is available
on Mac OS X and FreeBSD).


If you cannot come along, please make sure that our customers
know about this conference, and strongly encourage them to come.
It's rare for Australia to get an opportunity like this, so you
and they should make the most of it.


Dates:    15 - 17 July
Venue:    Queensland Brain Institute, University of Queensland

Conference homepage
http://au.sun.com/sunnews/events/2009/kernel/


Full abstracts
http://wikis.sun.com/display/KCA2009/KCA2009+Conference+Agenda


Registration page
https://www.conveneit.com/secure/sun/kernel_jul_09/



Pricing:
Students			$95
regular	price			$300

Corporate pricing		$195 per person
(please talk with Gabriele
DeCelis directly)


I look forward to seeing you at Kernel Conference Australia.


Best regards,
James C. McPherson
--
Senior Kernel Software Engineer, Solaris
Sun Microsystems
http://blogs.sun.com/jmcp	http://www.jmcp.homeunix.com/blog
Kernel Conference Australia - http://au.sun.com/sunnews/events/2009/kernel
From rick-freebsd2008 at kiwi-computer.com  Thu Jun 25 15:49:53 2009
From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty)
Date: Thu Jun 25 15:50:01 2009
Subject: dd copy of FreeBSD-7.2 won't boot
In-Reply-To: <2C153FCDAC3B43BDADDD1B28CED80036@uk.tiscali.intl>
References: <20090623032742.M54551@ezo.net>
	<2C153FCDAC3B43BDADDD1B28CED80036@uk.tiscali.intl>
Message-ID: <20090625154950.GA83146@keira.kiwi-computer.com>

On Wed, Jun 24, 2009 at 11:11:21PM +0100, krad wrote:
> Personally id stay away from dd. Create the partitions and file systems
> manually, and install the boot loader, then rsync the data across. It will
> be a lot faster in most cases, as unlike dd you wont be copying unused
> space. Something like this should do the job

I wouldn't say rsync is faster than dd, unless you have a lot of empty
space or are migrating across a network.  The nice thing about rsync is it
if you restart it, it picks up where it left off so to speak.  With dd you
have to add two arguments.

> Rsync -aPH --exclude=/mnt/** / /mnt
  ^

I often use:

	rsync -avHSPx / /mnt

The "x" means don't cross file system boundaries, which is generally what
you want when migrating file systems.

> I'm assuming you weren't migrating due to a bad disk

Actually using rsync to migrate a bad disk is preferrable over dd
(presuming you have no backup), since rsync will skip and warn you aboud
bad files, but "dd conv=noerror" could leave you in a bad situation if it
skips over critical metadata blocks.  I always try rsync first, with the
source filesystem mounted read-only, and if that fails I'll fall back to
dd.  Actually if you have the space and it was a bad disk, I'd probably dd
to a new disk or file, then mount that disk or file read-only, and then use
rsync.

-- Rick C. Petty
From joel at FreeBSD.org  Thu Jun 25 16:10:32 2009
From: joel at FreeBSD.org (Joel Dahl)
Date: Thu Jun 25 16:10:39 2009
Subject: 8 week projects
In-Reply-To: <1245861612.3381.6.camel@localhost.localdomain>
References: <1245861612.3381.6.camel@localhost.localdomain>
Message-ID: <4A439DAF.8010400@FreeBSD.org>

Sean Bruno skrev:
> My open source class this summer has a lot of people in it looking for 8
> week projects.
> 
> If you have a decently spec'd out project that a Junior/Senior CS
> student can accomplish, send me a link or pointer to it and I'll see if
> I can get the project some attention.

There are many TODO lists on the wiki (some may be outdated):

http://wiki.freebsd.org/Networking
http://wiki.freebsd.org/IPv6TODO
http://wiki.freebsd.org/BsnmpTODO
http://wiki.freebsd.org/Jails
http://wiki.freebsd.org/USBTODO
http://wiki.freebsd.org/SMPTODO

--
Joel

From jflowers at ezo.net  Thu Jun 25 17:49:46 2009
From: jflowers at ezo.net (jflowers)
Date: Thu Jun 25 17:49:54 2009
Subject: dd copy of FreeBSD-7.2 won't boot
In-Reply-To: <20090625154950.GA83146@keira.kiwi-computer.com>
References: <20090623032742.M54551@ezo.net>
	<2C153FCDAC3B43BDADDD1B28CED80036@uk.tiscali.intl>
	<20090625154950.GA83146@keira.kiwi-computer.com>
Message-ID: <20090625173404.M30669@ezo.net>

I wound up using kraduk's suggestion except for using a snapshot instead of a
live file system.  Because the source disk had suffered DMA errors and a few
files lost to SOFT UPDATE errors, I built a full system first and then let
rsync merge the two.

mount -u -o snapshot /snapshot/snap1 /
mdconfig -a -t vnode -f /snapshot/snap1 -u 4
mount -r /dev/md4 /mnt
mount /dev/ad4s1a /mnt_t
rsync -aPH --exclude=usr/dumps/** /mnt/* /mnt_t
umount /mnt
mdconfig -d -u 4
umount /mnt_t

repeated for all partitions (/ /tmp /var /usr).

Also had to edit transferred fstab to reflect new device name (ad4) and
install a new boot0.

mount /dev/ad4s1a /mnt_t
vi /mnt_t/etc/fstab
umount /mnt_t

fdisk -B -b /boot/boot0 /dev/ad4

18 GB took about 20 minutes to complete snapshots and another 25 minutes to
transfer via rsync.

Thanks for all the help.

On Thu, 25 Jun 2009 10:49:50 -0500, Rick C. Petty wrote
> On Wed, Jun 24, 2009 at 11:11:21PM +0100, krad wrote:
> > Personally id stay away from dd. Create the partitions and file systems
> > manually, and install the boot loader, then rsync the data across. It will
> > be a lot faster in most cases, as unlike dd you wont be copying unused
> > space. Something like this should do the job
> 
> I wouldn't say rsync is faster than dd, unless you have a lot of 
> empty space or are migrating across a network.  The nice thing about 
> rsync is it if you restart it, it picks up where it left off so to 
> speak.  With dd you have to add two arguments.
> 
> > Rsync -aPH --exclude=/mnt/** / /mnt
>   ^
> 
> I often use:
> 
> 	rsync -avHSPx / /mnt
> 
> The "x" means don't cross file system boundaries, which is generally 
> what you want when migrating file systems.
> 
> > I'm assuming you weren't migrating due to a bad disk
> 
> Actually using rsync to migrate a bad disk is preferrable over dd
> 
> (presuming you have no backup), since rsync will skip and warn you aboud
> bad files, but "dd conv=noerror" could leave you in a bad situation 
> if it skips over critical metadata blocks.  I always try rsync first,
>  with the source filesystem mounted read-only, and if that fails 
> I'll fall back to dd.  Actually if you have the space and it was a 
> bad disk, I'd probably dd to a new disk or file, then mount that 
> disk or file read-only, and then use rsync.
> 
> -- Rick C. Petty
> 
> -- 
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.


--
Jim Flowers <jflowers@ezo.net>

From alexbestms at math.uni-muenster.de  Thu Jun 25 22:25:33 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Thu Jun 25 22:25:41 2009
Subject: linux syscall modify_ldt() returning wrong errno
Message-ID: <permail-20090625222531f0889e840000548e-a_best01@message-id.uni-muenster.de>

hi there,

i'm currently playing a bit with the linux test project (ltp). it seems the
linux syscall modify_ldt() isn't implemented correctly. the following code
should set errno to ENOSYS, but instead EINVAL is being returned:

int main(int ac, char **av)
{
        int lc;                 /* loop counter */
        char *msg;              /* message returned from parse_opts */

        void *ptr;
        int retval, func;

        int flag;
        int seg[4];

        /* parse standard options */
        if ((msg = parse_opts(ac, av, (option_t *) NULL, NULL)) != (char
        *)NULL) {
                tst_brkm(TBROK, cleanup, "OPTION PARSING ERROR - %s", msg);
         /*NOTREACHED*/}

        setup();                /* global setup */

        /* The following loop checks looping state if -i option given */
        for (lc = 0; TEST_LOOPING(lc); lc++) {

                /* reset Tst_count in case we are looping */
                Tst_count = 0;

//block1:
                /*
                 * Check for ENOSYS.
                 */
                tst_resm(TINFO, "Enter block 1");
                flag = 0;
                ptr = (void *)malloc(10);
                func = 100;
                retval = modify_ldt(func, ptr, sizeof(ptr));
                if (retval < 0) {
                        if (errno != ENOSYS) {
                                tst_resm(TFAIL, "modify_ldt() set invalid "
                                         "errno, expected ENOSYS, got: %d",
                                         errno);
                                flag = FAILED;
                        }

here's the way linux does it:
http://fxr.watson.org/fxr/source/arch/x86_64/kernel/ldt.c?v=linux-2.6;im=bigexcerpts#L233

and this is how the syscall was implemented in freebsd:
http://fxr.watson.org/fxr/source/i386/linux/linux_machdep.c#L861

here's the modify_ldt() manual:
http://linux.about.com/library/cmd/blcmdl2_modify_ldt.htm

cheers.

oh...and we're talking HEAD of course. ;-)
From jhb at freebsd.org  Fri Jun 26 14:52:45 2009
From: jhb at freebsd.org (John Baldwin)
Date: Fri Jun 26 14:53:01 2009
Subject: linux syscall modify_ldt() returning wrong errno
In-Reply-To: <permail-20090625222531f0889e840000548e-a_best01@message-id.uni-muenster.de>
References: <permail-20090625222531f0889e840000548e-a_best01@message-id.uni-muenster.de>
Message-ID: <200906261033.52478.jhb@freebsd.org>

On Thursday 25 June 2009 6:25:31 pm Alexander Best wrote:
> hi there,
> 
> i'm currently playing a bit with the linux test project (ltp). it seems the
> linux syscall modify_ldt() isn't implemented correctly. the following code
> should set errno to ENOSYS, but instead EINVAL is being returned:

It looks like this should fix it:

--- //depot/vendor/freebsd/src/sys/i386/linux/linux_machdep.c	2009/02/18 16:15:14
+++ //depot/user/jhb/acpipci/i386/linux/linux_machdep.c	2009/06/26 14:32:55
@@ -866,9 +866,6 @@
 	union descriptor desc;
 	int size, written;
 
-	if (uap->ptr == NULL)
-		return (EINVAL);
-
 	switch (uap->func) {
 	case 0x00: /* read_ldt */
 		ldt.start = 0;
@@ -911,7 +908,7 @@
 		error = i386_set_ldt(td, &ldt, &desc);
 		break;
 	default:
-		error = EINVAL;
+		error = ENOSYS;
 		break;
 	}
 

-- 
John Baldwin
From bruce at cran.org.uk  Sat Jun 27 12:44:57 2009
From: bruce at cran.org.uk (Bruce Cran)
Date: Sat Jun 27 12:45:10 2009
Subject: ATA driver update for 7.2RELEASE available
In-Reply-To: <8EB69F68-8ED2-469C-B83E-2555A72630B4@deepcore.dk>
References: <8EB69F68-8ED2-469C-B83E-2555A72630B4@deepcore.dk>
Message-ID: <20090627132941.5d212017@tau.draftnet>

On Fri, 26 Jun 2009 22:11:08 +0200
S?ren Schmidt <sos@deepcore.dk> wrote:

> This is a total replacement of the ATA driver, modulerized as in - 
> current, but based on my WIP not from what might have happend to - 
> current since I gave up maintainership.

It's great to see the driver be modularised since removing unneeded
drivers (for example, on powerpc and embedded platforms) can save
~200KB.

Could you add some documentation for the modules in /sys/conf/NOTES
please?  I looked through the sources and put together the patch for
8.0 which is avaiable at
http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/133162 but it sounds
like some more drivers will need to be added for 7.2.

-- 
Bruce Cran
From alexbestms at math.uni-muenster.de  Sat Jun 27 20:00:41 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Sat Jun 27 20:00:48 2009
Subject: kdump crashing with linux binaries
Message-ID: <permail-20090627200038f7e55a9d00004aad-a_best01@message-id.uni-muenster.de>

hi there,

when ktrace is used with a linux binary and one runs kdump afterwards kdump
segfaults. there's a problem report here
http://www.freebsd.org/cgi/query-pr.cgi?pr=120055 with a patch.

of course linux_kdump should be used instead if the regular kdump, but it
still would be nice to have this fix in HEAD and get rid of this segfault.
could somebody commit this patch please?

cheers.
From sebastian.huber at embedded-brains.de  Sun Jun 28 10:24:47 2009
From: sebastian.huber at embedded-brains.de (Sebastian Huber)
Date: Sun Jun 28 10:24:54 2009
Subject: callout(9) and Giant lock
Message-ID: <4A4740B8.8090205@embedded-brains.de>

Hi,

suppose that a certain time event triggered several callout functions. 
What happens if the first of these callout functions blocks on the Giant 
lock? Does this delay all further callout functions until the Giant lock 
is available for the first callout function? What happens if one of the 
callout function blocks forever? Does this deadlock the system?

CU

-- 
Sebastian Huber, Embedded Brains GmbH

Address : Obere Lagerstr. 30, D-82178 Puchheim, Germany
Phone   : +49 89 18 90 80 79-6
Fax     : +49 89 18 90 80 79-9
E-Mail  : sebastian.huber@embedded-brains.de
PGP     : Public key available on request

Diese Nachricht ist keine gesch?ftliche Mitteilung im Sinne des EHUG.

From kpielorz_lst at tdx.co.uk  Sun Jun 28 15:43:55 2009
From: kpielorz_lst at tdx.co.uk (Karl Pielorz)
Date: Sun Jun 28 15:44:02 2009
Subject: ata 'Flush Cache' errors, on non-failing disk?
Message-ID: <20E145B15D43DBD9A741F1DB@Octa64>


Hi,

I've recently updated my amd64 system from 6.4 to 7.2-Stable - this works 
fine, but I've started picking up errors on the console:

  ad36: TIMEOUT - FLUSHCACHE retrying (1 retry left)

The drive (an WD5000AAKS) appears healthy - SMART reports no errors, or 
problems - and the timeouts only appear when that drive is 'being hammered' 
by write requests (e.g. during ZFS re-silvering to it)

The Western-Digi drive doctor CD/ISO runs a full test, and reports no 
problems (in that machine, with that drive).

I did find a number of posts, such as:

 <http://lists.freebsd.org/pipermail/freebsd-current/2009-April/005939.html>

Which point to the default timeout for the ATA flushcache command being 5 
seconds, when perhaps it should be 30...

But the code in 7.2-STABLE bears no resemblance to the code that the patch 
is for - so I'm guessing things have moved on since then...

Is there anywhere I might apply a similar patch to up the timeout to see if 
that cures the problem?

The only mentions of ATA_FLUSHCACHE appears to be calls to "ata_controlcmd( 
xxxx, ATA_FLUSHCACHE, 0, 0, 0);" - "ata_controlcmd" in turn seems to set a 
request timeout of '1' - but I can't tell if that's a timeout of 1 second, 
1 tick, or 1 what - or if it's a timeout for adding the command to the 
queue, or actually a timeout for executing that command...

Is upping that request timeout conditionally for cache flushes likely to 
have the effect I'm looking for?


-Kp
From rwatson at FreeBSD.org  Sun Jun 28 15:57:06 2009
From: rwatson at FreeBSD.org (Robert Watson)
Date: Sun Jun 28 15:57:12 2009
Subject: callout(9) and Giant lock
In-Reply-To: <4A4740B8.8090205@embedded-brains.de>
References: <4A4740B8.8090205@embedded-brains.de>
Message-ID: <alpine.BSF.2.00.0906281653160.69417@fledge.watson.org>


On Sun, 28 Jun 2009, Sebastian Huber wrote:

> suppose that a certain time event triggered several callout functions. What 
> happens if the first of these callout functions blocks on the Giant lock? 
> Does this delay all further callout functions until the Giant lock is 
> available for the first callout function? What happens if one of the callout 
> function blocks forever? Does this deadlock the system?

Callouts are marked as MPSAFE or non-MPSAFE when registered.  If non-MPSAFE, 
we will acquire Giant automatically for the callout, but I believe we'll also 
try and sort non-MPSAFE callouts behind MPSAFE ones in execution order to 
minimize latency for MPSAFE callouts.  Most callouts acquire locks of some 
sort, and stalling any callout indefinitely will stall the entire callout 
thread indefinitely, which in turn could lead to a variety of odd behaviors 
and potentially (although not necessarily) deadlock.

In general, we do not allow callouts to "block", however, in the sense that 
with INVARIANTS enabled we will actually panic if a callout tries to call 
msleep() or related functions.  Likewise, if another thread sleeps while 
holding Giant, it will automatically release it when it sleeps.

Relatively few kernel subsystems use Giant at this point, FYI, and even fewer 
in FreeBSD 8.  One of our goals for FreeBSD 9 is to eliminate all last 
remaining references to the Giant lock in the kernel.

Robert N M Watson
Computer Laboratory
University of Cambridge
From wojtek at wojtek.tensor.gdynia.pl  Sun Jun 28 17:39:42 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Sun Jun 28 17:39:48 2009
Subject: large pages (amd64)
Message-ID: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>

i enabled
vm.pmap.pg_ps_enabled: 1


could you please explain what exactly this values means?
because i don't understand why promotions-demotions!=mappings

vm.pmap.pde.promotions: 2703
vm.pmap.pde.p_failures: 6290
vm.pmap.pde.mappings: 610
vm.pmap.pde.demotions: 289




other question - tried enabling it on my i386 laptop (256 megs ram), 
always mappings==0, while promitions>demotions>0.

certainly there are apps that could be put on big pages, gimp editing 40MB 
bitmap for example
From msnkipa at mail.ru  Sun Jun 28 18:20:02 2009
From: msnkipa at mail.ru (=?koi8-r?Q?=ED=C9=C8=C1=C9=CC_=EB=C9=D0=C1?=)
Date: Sun Jun 28 18:20:09 2009
Subject: Can I bind POSIX thread to cpu core?
Message-ID: <E1MKyyu-0002jv-00.msnkipa-mail-ru@f103.mail.ru>

I have system with 4 core cpu. How can I bind POSIX thread to the one core? I mean that this thread 
can be executed on the fixed core.
From aryeh.friedman at gmail.com  Sun Jun 28 20:23:58 2009
From: aryeh.friedman at gmail.com (Aryeh M. Friedman)
Date: Sun Jun 28 20:24:05 2009
Subject: dynamic and static DNS in the samie name server
Message-ID: <4A47CC07.1080506@gmail.com>

We have a name server set up that contains all our static IP's (in 
office but not for remote workers because they all have dynamic 
IP's).... we currently use dns/no-ip2 to manage this  but I was 
wondering if it is possible to (easily) configure our internal 
nameserver to do this?
From dnelson at allantgroup.com  Sun Jun 28 20:44:52 2009
From: dnelson at allantgroup.com (Dan Nelson)
Date: Sun Jun 28 20:45:17 2009
Subject: Can I bind POSIX thread to cpu core?
In-Reply-To: <E1MKyyu-0002jv-00.msnkipa-mail-ru@f103.mail.ru>
References: <E1MKyyu-0002jv-00.msnkipa-mail-ru@f103.mail.ru>
Message-ID: <20090628204446.GI76275@dan.emsphone.com>

In the last episode (Jun 28):
> I have system with 4 core cpu. How can I bind POSIX thread to the one
> core?  I mean that this thread can be executed on the fixed core.

See the cpuset(2) and cpuset_setaffinity(2) manpages.  Something like this
should work:

#include <err.h>
#include <stdio.h>
#include <sys/param.h>
#include <sys/cpuset.h>

int main(void)
{
	int i;
	cpuset_t myset;

	/* Get CPU mask for the current thread */
	if (cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1, sizeof(myset), &myset) == -1)
		err(1, "getaffinity failed");

	/* Find first available CPU - don't assume CPU0 is always available */
	for (i = 0; i < CPU_SETSIZE; i++)
		if (CPU_ISSET(i, &myset))
			break; 

	if (i == CPU_SETSIZE)
		err(1, "Not allowed to run on any CPUs?  How did I print this, then?");

	/* Set new CPU mask */
	printf ("Setting affinity to CPU %d\n", i);
	CPU_ZERO(&myset);
	CPU_SET(i, &myset);

	if (cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1, sizeof(myset), &myset) == -1)
		warn("setaffinity failed");

	/* Do CPU-intensive stuff here */
	return 0;
}


-- 
	Dan Nelson
	dnelson@allantgroup.com
From aryeh.friedman at gmail.com  Sun Jun 28 21:37:56 2009
From: aryeh.friedman at gmail.com (Aryeh M. Friedman)
Date: Sun Jun 28 21:38:02 2009
Subject: dynamic and static DNS in the samie name server
In-Reply-To: <20090628170140.7169d64b@bhuda.mired.org>
References: <4A47CC07.1080506@gmail.com>
	<20090628170140.7169d64b@bhuda.mired.org>
Message-ID: <4A47E2B3.8040103@gmail.com>

Mike Meyer wrote:
> On Sun, 28 Jun 2009 16:01:11 -0400
> "Aryeh M. Friedman" <aryeh.friedman@gmail.com> wrote:
>
>   
>> We have a name server set up that contains all our static IP's (in 
>> office but not for remote workers because they all have dynamic 
>> IP's).... we currently use dns/no-ip2 to manage this  but I was 
>> wondering if it is possible to (easily) configure our internal 
>> nameserver to do this?
>>     
>
> This question is really more appropriate for -questions than
> -hackers. If you asked on -questions and didn't get an answer there or
> someone there pointed you here, you should let us know: it's always
> wise to indicate previous research.
>
> You also didn't say what you were using for your internal name server,
> which makes the matter of "easily" hard to answer. If you're a bind
> expert, it's easy. If you're not a bind expert but using bind, it
> probably isn't. Personally, I use dns/dnsmasq for such situations, as
> that's pretty much what it's designed to do. It's also very
> light-weight, sufficiently so that the various FOSS router builds
> include it.
>
> 	<mike
>
>   
Thanks for attempting to help me find the right place to post but in my 
experience -questions@ tends to be about "standard" usage not 
"non-standard" usage questions thus I asked here... yes I am using BIND 
(the base system one)
From alan.l.cox at gmail.com  Sun Jun 28 22:16:03 2009
From: alan.l.cox at gmail.com (Alan Cox)
Date: Sun Jun 28 22:16:10 2009
Subject: large pages (amd64)
In-Reply-To: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
Message-ID: <ca3526250906281451g4441441bt3718d37dd848590a@mail.gmail.com>

On Sun, Jun 28, 2009 at 12:36 PM, Wojciech Puchar <
wojtek@wojtek.tensor.gdynia.pl> wrote:

> i enabled
> vm.pmap.pg_ps_enabled: 1
>
>
> could you please explain what exactly this values means?
> because i don't understand why promotions-demotions!=mappings
>

"mappings" is not what you seem to think it is.  vm.pmap.pde.mappings is the
number of 2/4MB page mappings that are created directly and not through the
incremental promotion process.  For example, it counts the 2/4MB page
mappings that are created when the text segment of a large executable, e.g.,
gcc, is pre-faulted at startup or when a graphics card's frame buffer is
mmap()ed.

Moreover, not every promoted mapping is demoted.  A promoted mapping may be
destroyed without demotion, for example, when a process exits.  This is, in
fact, the ideal case because it is cheaper to destroy a single 2/4MB page
mapping instead of 512 or 1024 4KB page mappings.


>
> vm.pmap.pde.promotions: 2703
> vm.pmap.pde.p_failures: 6290
> vm.pmap.pde.mappings: 610
> vm.pmap.pde.demotions: 289
>
>
>
>
> other question - tried enabling it on my i386 laptop (256 megs ram), always
> mappings==0, while promitions>demotions>0.
>

The default starting address for executables on i386 is not aligned to a
2/4MB page boundary.  Hence, "mappings" are much less likely to occur.


> certainly there are apps that could be put on big pages, gimp editing 40MB
> bitmap for example


Regards,
Alan

<http://lists.freebsd.org/mailman/listinfo/freebsd-hackers><freebsd-hackers-unsubscribe@freebsd.org>
From wojtek at wojtek.tensor.gdynia.pl  Sun Jun 28 22:28:48 2009
From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar)
Date: Sun Jun 28 22:28:56 2009
Subject: large pages (amd64)
In-Reply-To: <ca3526250906281451g4441441bt3718d37dd848590a@mail.gmail.com>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
	<ca3526250906281451g4441441bt3718d37dd848590a@mail.gmail.com>
Message-ID: <alpine.BSF.2.00.0906290024540.37501@wojtek.tensor.gdynia.pl>

>
>       other question - tried enabling it on my i386 laptop (256 megs ram), always
>       mappings==0, while promitions>demotions>0.
> 
> 
> The default starting address for executables on i386 is not aligned to a 2/4MB page boundary. 
> Hence, "mappings" are much less likely to occur.
> 
>
>       certainly there are apps that could be put on big pages, gimp editing 40MB bitmap for
>       example
> 
> 
> Regards,
> Alan
> 
> 
> 
how can i check how much (or maybe - what processes) 2MB pages are 
actually allocated?
From alc at cs.rice.edu  Mon Jun 29 00:10:18 2009
From: alc at cs.rice.edu (Alan Cox)
Date: Mon Jun 29 00:10:27 2009
Subject: large pages (amd64)
In-Reply-To: <alpine.BSF.2.00.0906290024540.37501@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
	<ca3526250906281451g4441441bt3718d37dd848590a@mail.gmail.com>
	<alpine.BSF.2.00.0906290024540.37501@wojtek.tensor.gdynia.pl>
Message-ID: <4A47FFBD.3070900@cs.rice.edu>

Wojciech Puchar wrote:
>>
>>       other question - tried enabling it on my i386 laptop (256 megs 
>> ram), always
>>       mappings==0, while promitions>demotions>0.
>>
>>
>> The default starting address for executables on i386 is not aligned 
>> to a 2/4MB page boundary. Hence, "mappings" are much less likely to 
>> occur.
>>
>>
>>       certainly there are apps that could be put on big pages, gimp 
>> editing 40MB bitmap for
>>       example
>>
>>
>> Regards,
>> Alan
>>
>>
>>
> how can i check how much (or maybe - what processes) 2MB pages are 
> actually allocated?

I'm afraid that you can't with great precision.  For a given program 
execution, on an otherwise idle machine, you can only estimate the 
number by looking at the change in the quantity "promotions + mappings - 
demotions" before, during, and after the program execution.

A program can call mincore(2) in order to determine if a virtual address 
is part of a 2 or 4MB virtual page.

Alan

From nhoyle at hoyletech.com  Mon Jun 29 00:27:09 2009
From: nhoyle at hoyletech.com (Nathanael Hoyle)
Date: Mon Jun 29 00:27:16 2009
Subject: large pages (amd64)
In-Reply-To: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
Message-ID: <4A480760.50705@hoyletech.com>

Wojciech Puchar wrote:
> i enabled
> vm.pmap.pg_ps_enabled: 1
>
>
> could you please explain what exactly this values means?
> because i don't understand why promotions-demotions!=mappings
>
> vm.pmap.pde.promotions: 2703
> vm.pmap.pde.p_failures: 6290
> vm.pmap.pde.mappings: 610
> vm.pmap.pde.demotions: 289
>
>
>
>
> other question - tried enabling it on my i386 laptop (256 megs ram), 
> always mappings==0, while promitions>demotions>0.
>
> certainly there are apps that could be put on big pages, gimp editing 
> 40MB bitmap for example

Just to be clear, since you say i386 (I presume you mean architecture), 
I believe the Physical Address Extensions which allowed 2MB Page Size 
bit to be set was introduced with Pentium Pro. Processors prior to this 
were limited to standard 4KB pages.

-Nathanael
From joerg at britannica.bec.de  Mon Jun 29 00:35:20 2009
From: joerg at britannica.bec.de (Joerg Sonnenberger)
Date: Mon Jun 29 00:35:27 2009
Subject: large pages (amd64)
In-Reply-To: <4A480760.50705@hoyletech.com>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
	<4A480760.50705@hoyletech.com>
Message-ID: <20090629003522.GA18543@britannica.bec.de>

On Sun, Jun 28, 2009 at 08:14:24PM -0400, Nathanael Hoyle wrote:
> Just to be clear, since you say i386 (I presume you mean architecture),  
> I believe the Physical Address Extensions which allowed 2MB Page Size  
> bit to be set was introduced with Pentium Pro. Processors prior to this  
> were limited to standard 4KB pages.

PSE != PAE != PSE36. Pentium Pro sounds about right though.

Joerg
From alan.l.cox at gmail.com  Mon Jun 29 00:41:16 2009
From: alan.l.cox at gmail.com (Alan Cox)
Date: Mon Jun 29 00:41:23 2009
Subject: large pages (amd64)
In-Reply-To: <4A480760.50705@hoyletech.com>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
	<4A480760.50705@hoyletech.com>
Message-ID: <ca3526250906281741h5c6c9407vdf9ea849330023ed@mail.gmail.com>

On Sun, Jun 28, 2009 at 7:14 PM, Nathanael Hoyle <nhoyle@hoyletech.com>wrote:

> Wojciech Puchar wrote:
>
>> i enabled
>> vm.pmap.pg_ps_enabled: 1
>>
>>
>> could you please explain what exactly this values means?
>> because i don't understand why promotions-demotions!=mappings
>>
>> vm.pmap.pde.promotions: 2703
>> vm.pmap.pde.p_failures: 6290
>> vm.pmap.pde.mappings: 610
>> vm.pmap.pde.demotions: 289
>>
>>
>>
>>
>> other question - tried enabling it on my i386 laptop (256 megs ram),
>> always mappings==0, while promitions>demotions>0.
>>
>> certainly there are apps that could be put on big pages, gimp editing 40MB
>> bitmap for example
>>
>
> Just to be clear, since you say i386 (I presume you mean architecture), I
> believe the Physical Address Extensions which allowed 2MB Page Size bit to
> be set was introduced with Pentium Pro. Processors prior to this were
> limited to standard 4KB pages.
>

No.  Many of those processors supported 4MB pages.

Regards,
Alan
From nhoyle at hoyletech.com  Mon Jun 29 00:54:30 2009
From: nhoyle at hoyletech.com (Nathanael Hoyle)
Date: Mon Jun 29 00:54:36 2009
Subject: large pages (amd64)
In-Reply-To: <ca3526250906281741h5c6c9407vdf9ea849330023ed@mail.gmail.com>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>	<4A480760.50705@hoyletech.com>
	<ca3526250906281741h5c6c9407vdf9ea849330023ed@mail.gmail.com>
Message-ID: <4A4810BB.1010007@hoyletech.com>

Alan Cox wrote:
> On Sun, Jun 28, 2009 at 7:14 PM, Nathanael Hoyle <nhoyle@hoyletech.com>wrote:
>
>   
>> Wojciech Puchar wrote:
>>
>>     
>>> i enabled
>>> vm.pmap.pg_ps_enabled: 1
>>>
>>>
>>> could you please explain what exactly this values means?
>>> because i don't understand why promotions-demotions!=mappings
>>>
>>> vm.pmap.pde.promotions: 2703
>>> vm.pmap.pde.p_failures: 6290
>>> vm.pmap.pde.mappings: 610
>>> vm.pmap.pde.demotions: 289
>>>
>>>
>>>
>>>
>>> other question - tried enabling it on my i386 laptop (256 megs ram),
>>> always mappings==0, while promitions>demotions>0.
>>>
>>> certainly there are apps that could be put on big pages, gimp editing 40MB
>>> bitmap for example
>>>
>>>       
>> Just to be clear, since you say i386 (I presume you mean architecture), I
>> believe the Physical Address Extensions which allowed 2MB Page Size bit to
>> be set was introduced with Pentium Pro. Processors prior to this were
>> limited to standard 4KB pages.
>>
>>     
>
> No.  Many of those processors supported 4MB pages.
>
> Regards,
> Alan
> _______________________________________________
>   
Having been corrected by both you and Joerg (thank you!), I went back to 
re-verify my understanding. It appears that while I was slightly mixing 
PAE in with PSE, PSE support for 4MB pages was introduced 'silently' 
with the Pentium, and documented first with the Pentium Pro.  I haven't 
found anything that points to earlier inclusion. Certainly the 80386 
processor specifically, I am fairly confident would be limited to the 
4KB pages.

Agreed? Or are you aware of earlier usage than the Pentium for 4MB pages?

Thanks,
-Nathanael
From sos at deepcore.dk  Fri Jun 26 20:24:10 2009
From: sos at deepcore.dk (=?ISO-8859-1?Q?S=F8ren_Schmidt?=)
Date: Mon Jun 29 01:00:41 2009
Subject: ATA driver update for 7.2RELEASE available
Message-ID: <8EB69F68-8ED2-469C-B83E-2555A72630B4@deepcore.dk>

Over the past months I've gotten huge amounts of requests for ATA  
related things, so I've whipped up what I use here for FreeBSD 7.2- 
Release.

This is a total replacement of the ATA driver, modulerized as in - 
current, but based on my WIP not from what might have happend to - 
current since I gave up maintainership.

There is a number of new chipsets supported in here that I dont think  
is in any of the official sources (havn't checked in quite some time  
though), mostly newish ATI, nVidia and Marvell chips (yeah those  
buggers with both ATA & SATA ports).

You can find and install.sh script and a tarball of the needed files at:

http://www.deepcore.dk/pub/ATA

Put both in /usr/src and run install.sh.

As I dont subscribe to any of the mailing lists nor does my  
FreeBSD.org mail address seem to work anymore, you will need to reply  
to me directly if needed.

As always - Enjoy!

-S?ren
--









From bvowk at math.ualberta.ca  Fri Jun 26 23:39:04 2009
From: bvowk at math.ualberta.ca (Barkley Vowk)
Date: Mon Jun 29 01:00:56 2009
Subject: rpc.yppasswdd fails in 7.2?
Message-ID: <alpine.BSF.2.00.0906261648070.95627@3jane.math.ualberta.ca>

I've got a 7.2 box acting as NIS master, running rpc.yppasswdd. When 
remote users try to change their passwords they see:

testing1# passwd mytest
Changing NIS password for mytest
Old Password:
New Password:
Retype New Password:
passwd: pam_chauthtok(): error in service module

On the server I get this in the logs:
rpc.yppasswdd[36858]: pw_mkdb() failed

I tracked the error message down to line 620 of yppasswdd_server.c, which
lead me to libutil/pw_util.c.

It looks like waitpid in mk_pwdb is returning the error "No child 
processes", which then makes mk_pwdb return an error. But the child does 
run and do it's job as expected. If you edit yppasswdd_server.c to ignore 
the return value of mk_pwdb, passwords are changed as expected.


From nhoyle at hoyletech.com  Mon Jun 29 01:03:51 2009
From: nhoyle at hoyletech.com (Nathanael Hoyle)
Date: Mon Jun 29 01:03:57 2009
Subject: large pages (amd64)
In-Reply-To: <20090629003522.GA18543@britannica.bec.de>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>	<4A480760.50705@hoyletech.com>
	<20090629003522.GA18543@britannica.bec.de>
Message-ID: <4A4812F0.2080208@hoyletech.com>

Joerg Sonnenberger wrote:
> On Sun, Jun 28, 2009 at 08:14:24PM -0400, Nathanael Hoyle wrote:
>   
>> Just to be clear, since you say i386 (I presume you mean architecture),  
>> I believe the Physical Address Extensions which allowed 2MB Page Size  
>> bit to be set was introduced with Pentium Pro. Processors prior to this  
>> were limited to standard 4KB pages.
>>     
>
> PSE != PAE != PSE36. Pentium Pro sounds about right though.
>
> Joerg
>   
Thanks for the clarification/correction. PSE does not depend on PAE, 
though both were introduced at roughly the same time (appears PSE was 
introduced silently in Pentium, officially in Pentium Pro). I definitely 
wasn't thinking of PSE36, which was the Pentium III.

-Nathanael
From alc at cs.rice.edu  Mon Jun 29 05:59:16 2009
From: alc at cs.rice.edu (Alan Cox)
Date: Mon Jun 29 05:59:23 2009
Subject: large pages (amd64)
In-Reply-To: <4A4810BB.1010007@hoyletech.com>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>	<4A480760.50705@hoyletech.com>
	<ca3526250906281741h5c6c9407vdf9ea849330023ed@mail.gmail.com>
	<4A4810BB.1010007@hoyletech.com>
Message-ID: <4A48582B.8050009@cs.rice.edu>

Nathanael Hoyle wrote:
[snip]
> Having been corrected by both you and Joerg (thank you!), I went back 
> to re-verify my understanding. It appears that while I was slightly 
> mixing PAE in with PSE, PSE support for 4MB pages was introduced 
> 'silently' with the Pentium, and documented first with the Pentium 
> Pro.  I haven't found anything that points to earlier inclusion. 
> Certainly the 80386 processor specifically, I am fairly confident 
> would be limited to the 4KB pages.
>
> Agreed? Or are you aware of earlier usage than the Pentium for 4MB pages?
>

Yes, I agree.  I'm not aware of 4MB page support before the Pentium.

Regards,
Alan

From kostikbel at gmail.com  Mon Jun 29 10:02:19 2009
From: kostikbel at gmail.com (Kostik Belousov)
Date: Mon Jun 29 10:02:27 2009
Subject: rpc.yppasswdd fails in 7.2?
In-Reply-To: <alpine.BSF.2.00.0906261648070.95627@3jane.math.ualberta.ca>
References: <alpine.BSF.2.00.0906261648070.95627@3jane.math.ualberta.ca>
Message-ID: <20090629094923.GU2884@deviant.kiev.zoral.com.ua>

On Fri, Jun 26, 2009 at 04:59:29PM -0600, Barkley Vowk wrote:
> I've got a 7.2 box acting as NIS master, running rpc.yppasswdd. When 
> remote users try to change their passwords they see:
> 
> testing1# passwd mytest
> Changing NIS password for mytest
> Old Password:
> New Password:
> Retype New Password:
> passwd: pam_chauthtok(): error in service module
> 
> On the server I get this in the logs:
> rpc.yppasswdd[36858]: pw_mkdb() failed
> 
> I tracked the error message down to line 620 of yppasswdd_server.c, which
> lead me to libutil/pw_util.c.
> 
> It looks like waitpid in mk_pwdb is returning the error "No child 
> processes", which then makes mk_pwdb return an error. But the child does 
> run and do it's job as expected. If you edit yppasswdd_server.c to ignore 
> the return value of mk_pwdb, passwords are changed as expected.

I think this is a side-effect of r184459/r186756, that made rpc.yppasswdd to
ignore SIGCHLD.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090629/1c5529ac/attachment.pgp
From rafan at infor.org  Mon Jun 29 11:40:32 2009
From: rafan at infor.org (Rong-En Fan)
Date: Mon Jun 29 12:03:09 2009
Subject: rpc.yppasswdd fails in 7.2?
In-Reply-To: <20090629094923.GU2884@deviant.kiev.zoral.com.ua>
References: <alpine.BSF.2.00.0906261648070.95627@3jane.math.ualberta.ca>
	<20090629094923.GU2884@deviant.kiev.zoral.com.ua>
Message-ID: <20090629112144.GD27380@svm.csie.ntu.edu.tw>

On Mon, Jun 29, 2009 at 12:49:23PM +0300, Kostik Belousov wrote:
> On Fri, Jun 26, 2009 at 04:59:29PM -0600, Barkley Vowk wrote:
> > I've got a 7.2 box acting as NIS master, running rpc.yppasswdd. When 
> > remote users try to change their passwords they see:
> > 
> > testing1# passwd mytest
> > Changing NIS password for mytest
> > Old Password:
> > New Password:
> > Retype New Password:
> > passwd: pam_chauthtok(): error in service module
> > 
> > On the server I get this in the logs:
> > rpc.yppasswdd[36858]: pw_mkdb() failed
> > 
> > I tracked the error message down to line 620 of yppasswdd_server.c, which
> > lead me to libutil/pw_util.c.
> > 
> > It looks like waitpid in mk_pwdb is returning the error "No child 
> > processes", which then makes mk_pwdb return an error. But the child does 
> > run and do it's job as expected. If you edit yppasswdd_server.c to ignore 
> > the return value of mk_pwdb, passwords are changed as expected.
> 
> I think this is a side-effect of r184459/r186756, that made rpc.yppasswdd to
> ignore SIGCHLD.

It only happens when the master uses /etc/master.passwd instead of
/var/yp/master.passwd. 

The zombie mentioned in the commit log was yppwupdate. Two possible
soluutions are: we wait until the push is succeed or we use a double
fork. Personally, I prefer the former approach.

Regards,
Rong-En Fan


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090629/dce09e3c/attachment.pgp
From mel.flynn+fbsd.hackers at mailing.thruhere.net  Mon Jun 29 21:17:14 2009
From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn)
Date: Mon Jun 29 21:17:20 2009
Subject: large pages (amd64)
In-Reply-To: <4A47FFBD.3070900@cs.rice.edu>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
	<alpine.BSF.2.00.0906290024540.37501@wojtek.tensor.gdynia.pl>
	<4A47FFBD.3070900@cs.rice.edu>
Message-ID: <200906291317.12040.mel.flynn+fbsd.hackers@mailing.thruhere.net>

On Sunday 28 June 2009 15:41:49 Alan Cox wrote:
> Wojciech Puchar wrote:

> > how can i check how much (or maybe - what processes) 2MB pages are
> > actually allocated?
>
> I'm afraid that you can't with great precision.  For a given program
> execution, on an otherwise idle machine, you can only estimate the
> number by looking at the change in the quantity "promotions + mappings -
> demotions" before, during, and after the program execution.
>
> A program can call mincore(2) in order to determine if a virtual address
> is part of a 2 or 4MB virtual page.

Would it be possible to expose the super page count as kve_super in the 
kinfo_vmentry struct so that procstat can show this information? If only to 
determine if one is using the feature and possibly benefiting from it.

It looks like sys/kern/kern_proc.c could call mincore around the loop at line 
1601 (rev 194498), but I know nothing about the vm subsystem to know the 
implications or locking involved. There's still 16 bytes of spare to consume, 
in the kve_vminfo struct though ;)
-- 
Mel
From xorquewasp at googlemail.com  Mon Jun 29 21:36:55 2009
From: xorquewasp at googlemail.com (xorquewasp@googlemail.com)
Date: Mon Jun 29 21:37:02 2009
Subject: bootstrapping gnat GCC on amd64
Message-ID: <20090629213652.GA28392@logik.internal.network>

On 2009-06-29 17:27:40, Daniel Eischen wrote:
> How's this all going?
>
> I'm updating lang/gnat to gnat-2009 right now.  In the
> GNAT sources, it has build instructions including how
> to build the cross - see below.

'Lo,

Thanks for the info. GPL 2009 was also on my list, as was
SPARK (apparently that's going to take a bit of work as it
doesn't work out of the box with any freely available prolog).

I'm currently stalled trying to get the test suite to run
so that I can submit a couple of patches back upstream to
the GCC project. Once they're in, I can start working on
a lang/gnat-gcc44 port.

xw
From eischen at vigrid.com  Mon Jun 29 21:47:35 2009
From: eischen at vigrid.com (Daniel Eischen)
Date: Mon Jun 29 21:47:42 2009
Subject: bootstrapping gnat GCC on amd64
In-Reply-To: <20090519135917.GA5391@logik.internal.network>
References: <20090505022151.GA32477@logik.internal.network>
	<Pine.GSO.4.64.0905042224180.16666@sea.ntplx.net>
	<20090506140325.GA69468@logik.internal.network>
	<Pine.GSO.4.64.0905061043420.26946@sea.ntplx.net>
	<20090506152222.GC69468@logik.internal.network>
	<20090508211022.GA37475@logik.internal.network>
	<20090518084831.GA95354@logik.internal.network>
	<Pine.GSO.4.64.0905181828110.20749@sea.ntplx.net>
	<20090519114548.GA8610@logik.internal.network>
	<Pine.GSO.4.64.0905190945330.25229@sea.ntplx.net>
	<20090519135917.GA5391@logik.internal.network>
Message-ID: <Pine.GSO.4.64.0906291725330.2980@sea.ntplx.net>

How's this all going?

I'm updating lang/gnat to gnat-2009 right now.  In the
GNAT sources, it has build instructions including how
to build the cross - see below.

-- 
DE

--------------------------------------
-- BUILDING GNAT - EXAMPLE SEQUENCE --
--------------------------------------

Supposing you have one tarball for the GCC sources and one for the GNAT
sources, here is a possible way to proceed:

0/ If not available on your system, build libgmp and libmpfr following
    build instructions from these packages.

1/ Create a root directory into which you'll later setup the source and
    build subdirectories:

    ~ $ mkdir GNAT
    ~ $ cd GNAT

2/ Setup an initial source tree from the GCC and the GNAT tarballs:

   ~/GNAT $ gzip -dc [...]/gcc-4.3*.tgz | tar xf -
   ~/GNAT $ mv gcc-4* src
   ~/GNAT $ cd src

   ~/GNAT/src $ gzip -dc [...]/gnat-xxx-src.tgz  | tar xf -
   ~/GNAT/src $ mv gnat-xxx-src/src/ada gcc
   ~/GNAT/src $ patch -p0 < gnat-xxx-src/src/gcc-43.dif
   ~/GNAT/src $ touch gcc/cstamp-h.in

3/ Prepare and configure the build tree:

   ~/GNAT/src $ mkdir ../obj; cd ../obj
   ~/GNAT/obj $ ../src/configure --enable-languages="c,ada" --disable-libada ...
   [see the GCC documentation for other relevant configure options]

4/ Build the compiler, run time and tools:

   Note that the use of "GNU Make" is required by the GCC Makefiles.

   ~/GNAT/obj $ make bootstrap
   ~/GNAT/obj $ make -C gcc gnatlib gnattools

   If you are building a cross compiler, you will need to replace
   the above sequence by something similar to:

   ~/GNAT/obj $ make
   ~/GNAT/obj $ make -C gcc gnatlib cross-gnattools ada.all.cross

5/ Install the whole package:

   ~/GNAT/obj $ make install

From alc at cs.rice.edu  Tue Jun 30 08:24:09 2009
From: alc at cs.rice.edu (Alan Cox)
Date: Tue Jun 30 08:24:17 2009
Subject: large pages (amd64)
In-Reply-To: <200906291317.12040.mel.flynn+fbsd.hackers@mailing.thruhere.net>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
	<alpine.BSF.2.00.0906290024540.37501@wojtek.tensor.gdynia.pl>
	<4A47FFBD.3070900@cs.rice.edu>
	<200906291317.12040.mel.flynn+fbsd.hackers@mailing.thruhere.net>
Message-ID: <4A49CBA0.1050806@cs.rice.edu>

Mel Flynn wrote:
> On Sunday 28 June 2009 15:41:49 Alan Cox wrote:
>   
>> Wojciech Puchar wrote:
>>     
>
>   
>>> how can i check how much (or maybe - what processes) 2MB pages are
>>> actually allocated?
>>>       
>> I'm afraid that you can't with great precision.  For a given program
>> execution, on an otherwise idle machine, you can only estimate the
>> number by looking at the change in the quantity "promotions + mappings -
>> demotions" before, during, and after the program execution.
>>
>> A program can call mincore(2) in order to determine if a virtual address
>> is part of a 2 or 4MB virtual page.
>>     
>
> Would it be possible to expose the super page count as kve_super in the 
> kinfo_vmentry struct so that procstat can show this information? If only to 
> determine if one is using the feature and possibly benefiting from it.
>
>   

Yes, I think so. 

> It looks like sys/kern/kern_proc.c could call mincore around the loop at line 
> 1601 (rev 194498), but I know nothing about the vm subsystem to know the 
> implications or locking involved. There's still 16 bytes of spare to consume, 
> in the kve_vminfo struct though ;)
>   

Yes, to start with, you could replace the call to pmap_extract() with a 
call to pmap_mincore() and export a Boolean to user space that says, 
"This region of the address space contains one or more superpage mappings."

Counting the number of superpages is a little trickier.  The problem 
being that pmap_mincore() doesn't tell you how large the underlying page 
is.  So, the loop at line 1601 can't easily tell where one superpage 
ends and the next 4KB page or superpage begins, making counting the 
number of superpages in a region a little tricky.  One possibility is to 
change pmap_mincore() to return the page size (or the logarithm of the 
page size) rather than a single bit.

If you want to give it a try, I'll be happy to help.  There aren't 
really any "implications" or synchronization issues that you need to 
worry about.

Alan



From malathiramya at gmail.com  Tue Jun 30 09:52:58 2009
From: malathiramya at gmail.com (malathi selvaraj)
Date: Tue Jun 30 09:53:17 2009
Subject: good afternoon to all
Message-ID: <fc09aacb0906300252s11a3d1e6g9ae90bb8f8e7346@mail.gmail.com>

i am not able to made rcpbind,
i get an error while making the rcpbind
like this

RPC: Unknown host
showmount: can't do exports rpc

Thanks in advance
-- 
S.MALATHI
From alexbestms at math.uni-muenster.de  Tue Jun 30 16:12:22 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Tue Jun 30 16:12:29 2009
Subject: c question: *printf'ing arrays
Message-ID: <permail-2009063016121880e26a0b000035b1-a_best01@message-id.uni-muenster.de>

hi there,

i need to output the header of a file to stdout. the header looks like this:

    struct Header
    {
        u_int8_t rom_entry[4];
        u_int8_t nintendo_logo[156];
        u_char game_title[12];
        u_char game_code[4];
        u_char maker_code[2];
        u_int8_t fixed_val;
        u_int8_t unit_code;
        u_int8_t device_type;
        u_int8_t reserved_area1[7];
        u_int8_t software_version;
        u_int8_t complement_check;
        u_int8_t reserved_area2;
        u_int8_t ram_entry[4];
        u_int8_t boot_mode;
        u_int8_t slave_id;
        u_int8_t unused_area[26];
        u_int8_t joybus_entry[4];
    };

if there a way to use printf or some other variant of *printf without using
sizeof()-loops for all the arrays?

cheers.
From tevans.uk at googlemail.com  Tue Jun 30 16:49:34 2009
From: tevans.uk at googlemail.com (Tom Evans)
Date: Tue Jun 30 16:49:41 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-2009063016121880e26a0b000035b1-a_best01@message-id.uni-muenster.de>
References: <permail-2009063016121880e26a0b000035b1-a_best01@message-id.uni-muenster.de>
Message-ID: <1246380570.2437.219.camel@strangepork.london.mintel.ad>

On Tue, 2009-06-30 at 18:12 +0200, Alexander Best wrote:
> hi there,
> 
> i need to output the header of a file to stdout. the header looks like this:
> 
>     struct Header
>     {
>         u_int8_t rom_entry[4];
>         u_int8_t nintendo_logo[156];
>         u_char game_title[12];
>         u_char game_code[4];
>         u_char maker_code[2];
>         u_int8_t fixed_val;
>         u_int8_t unit_code;
>         u_int8_t device_type;
>         u_int8_t reserved_area1[7];
>         u_int8_t software_version;
>         u_int8_t complement_check;
>         u_int8_t reserved_area2;
>         u_int8_t ram_entry[4];
>         u_int8_t boot_mode;
>         u_int8_t slave_id;
>         u_int8_t unused_area[26];
>         u_int8_t joybus_entry[4];
>     };
> 
> if there a way to use printf or some other variant of *printf without using
> sizeof()-loops for all the arrays?
> 
> cheers.

None of your arrays are dynamically sized, so surely 
        write(fd, &hdr, sizeof(Header));
would do the trick?

Cheers

Tom

From alexbestms at math.uni-muenster.de  Tue Jun 30 17:10:46 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Tue Jun 30 17:10:52 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <1246380570.2437.219.camel@strangepork.london.mintel.ad>
Message-ID: <permail-20090630171044f7e55a9d0000757d-a_best01@message-id.uni-muenster.de>

thanks. but that simply dumps the contents of the struct to stdout. but since
most of the struct's contents aren't ascii the output isn't really of much
use.

cheers.

Tom Evans schrieb am 2009-06-30:
> On Tue, 2009-06-30 at 18:12 +0200, Alexander Best wrote:
> > hi there,

> > i need to output the header of a file to stdout. the header looks
> > like this:

> >     struct Header
> >     {
> >         u_int8_t rom_entry[4];
> >         u_int8_t nintendo_logo[156];
> >         u_char game_title[12];
> >         u_char game_code[4];
> >         u_char maker_code[2];
> >         u_int8_t fixed_val;
> >         u_int8_t unit_code;
> >         u_int8_t device_type;
> >         u_int8_t reserved_area1[7];
> >         u_int8_t software_version;
> >         u_int8_t complement_check;
> >         u_int8_t reserved_area2;
> >         u_int8_t ram_entry[4];
> >         u_int8_t boot_mode;
> >         u_int8_t slave_id;
> >         u_int8_t unused_area[26];
> >         u_int8_t joybus_entry[4];
> >     };

> > if there a way to use printf or some other variant of *printf
> > without using
> > sizeof()-loops for all the arrays?

> > cheers.

> None of your arrays are dynamically sized, so surely
>         write(fd, &hdr, sizeof(Header));
> would do the trick?

> Cheers

> Tom
From alexbestms at math.uni-muenster.de  Tue Jun 30 17:32:43 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Tue Jun 30 17:32:53 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <a2b6592c0906301016q6823983fq5c7d97587a62e291@mail.gmail.com>
Message-ID: <permail-20090630173241f7e55a9d000009a9-a_best01@message-id.uni-muenster.de>

that works, but i really want to have a pretty output to stdout. i guess i
have to stick with printf and use `for (i=0; i < sizeof(XXX); i++)` for each
array in the struct. just thought i could avoid it.

btw. `./my-program | hexdump` works, but if i do `./my-program > output`
output is being created, but is empty. is this normal?

cheers.

Igor Mozolevsky schrieb am 2009-06-30:
> 2009/6/30 Alexander Best <alexbestms@math.uni-muenster.de>:
> > thanks. but that simply dumps the contents of the struct to stdout.
> > but since
> > most of the struct's contents aren't ascii the output isn't really
> > of much
> > use.

> How about ./your-program | hexdump ?

> --
> Igor
From igor at hybrid-lab.co.uk  Tue Jun 30 17:41:45 2009
From: igor at hybrid-lab.co.uk (Igor Mozolevsky)
Date: Tue Jun 30 17:41:51 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-20090630171044f7e55a9d0000757d-a_best01@message-id.uni-muenster.de>
References: <1246380570.2437.219.camel@strangepork.london.mintel.ad> 
	<permail-20090630171044f7e55a9d0000757d-a_best01@message-id.uni-muenster.de>
Message-ID: <a2b6592c0906301016q6823983fq5c7d97587a62e291@mail.gmail.com>

2009/6/30 Alexander Best <alexbestms@math.uni-muenster.de>:
> thanks. but that simply dumps the contents of the struct to stdout. but since
> most of the struct's contents aren't ascii the output isn't really of much
> use.

How about ./your-program | hexdump ?

--
Igor
From igor at hybrid-lab.co.uk  Tue Jun 30 18:00:43 2009
From: igor at hybrid-lab.co.uk (Igor Mozolevsky)
Date: Tue Jun 30 18:00:51 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-20090630173241f7e55a9d000009a9-a_best01@message-id.uni-muenster.de>
References: <a2b6592c0906301016q6823983fq5c7d97587a62e291@mail.gmail.com> 
	<permail-20090630173241f7e55a9d000009a9-a_best01@message-id.uni-muenster.de>
Message-ID: <a2b6592c0906301100g55827437o66e09d6ca1269cdd@mail.gmail.com>

2009/6/30 Alexander Best <alexbestms@math.uni-muenster.de>:
> that works, but i really want to have a pretty output to stdout. i guess i
> have to stick with printf and use `for (i=0; i < sizeof(XXX); i++)` for each
> array in the struct. just thought i could avoid it.
>
> btw. `./my-program | hexdump` works, but if i do `./my-program > output`
> output is being created, but is empty. is this normal?

Depends if you output to stdout or stderr --- `>' redirects stdout.


Cheers,
--
Igor
From alexbestms at math.uni-muenster.de  Tue Jun 30 18:03:35 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Tue Jun 30 18:03:41 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <a2b6592c0906301100g55827437o66e09d6ca1269cdd@mail.gmail.com>
Message-ID: <permail-20090630180321f7e55a9d00002466-a_best01@message-id.uni-muenster.de>

should be stdout.


struct Header *hdr = rom;

int new_fd = open("/dev/stdout", O_RDWR);

printf("SIZE: %d\n",sizeof(*hdr));

write(new_fd, hdr, sizeof(*hdr));

close(new_fd);

Igor Mozolevsky schrieb am 2009-06-30:
> 2009/6/30 Alexander Best <alexbestms@math.uni-muenster.de>:
> > that works, but i really want to have a pretty output to stdout. i
> > guess i
> > have to stick with printf and use `for (i=0; i < sizeof(XXX); i++)`
> > for each
> > array in the struct. just thought i could avoid it.

> > btw. `./my-program | hexdump` works, but if i do `./my-program >
> > output`
> > output is being created, but is empty. is this normal?

> Depends if you output to stdout or stderr --- `>' redirects stdout.


> Cheers,
> --
> Igor
From rick-freebsd2008 at kiwi-computer.com  Tue Jun 30 18:11:23 2009
From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty)
Date: Tue Jun 30 18:11:30 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-20090630180321f7e55a9d00002466-a_best01@message-id.uni-muenster.de>
References: <a2b6592c0906301100g55827437o66e09d6ca1269cdd@mail.gmail.com>
	<permail-20090630180321f7e55a9d00002466-a_best01@message-id.uni-muenster.de>
Message-ID: <20090630181121.GA32665@keira.kiwi-computer.com>

On Tue, Jun 30, 2009 at 08:03:21PM +0200, Alexander Best wrote:
> should be stdout.
> 
> 
> struct Header *hdr = rom;
> 
> int new_fd = open("/dev/stdout", O_RDWR);
> printf("SIZE: %d\n",sizeof(*hdr));
> write(new_fd, hdr, sizeof(*hdr));
> close(new_fd);

Why are you reopening stdout?  It should already be open, so use
"fileno(stdout)" or just plain "STDOUT_FILENO" instead of "new_fd".

-- Rick C. Petty
From igor at hybrid-lab.co.uk  Tue Jun 30 18:19:45 2009
From: igor at hybrid-lab.co.uk (Igor Mozolevsky)
Date: Tue Jun 30 18:19:53 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-20090630180321f7e55a9d00002466-a_best01@message-id.uni-muenster.de>
References: <a2b6592c0906301100g55827437o66e09d6ca1269cdd@mail.gmail.com> 
	<permail-20090630180321f7e55a9d00002466-a_best01@message-id.uni-muenster.de>
Message-ID: <a2b6592c0906301119t4e4fe891ha07895c27b6efec0@mail.gmail.com>

2009/6/30 Alexander Best <alexbestms@math.uni-muenster.de>:
> should be stdout.
>
>
> struct Header *hdr = rom;
>
> int new_fd = open("/dev/stdout", O_RDWR);
>
> printf("SIZE: %d\n",sizeof(*hdr));
>
> write(new_fd, hdr, sizeof(*hdr));
>
> close(new_fd);

You should really be checking what open returns, opening /dev/stdout
for reading is a bit weird not sure if that would work, and most
likely it's already open... Just use fileno(...):-

#include <unistd.h>
#include <stdio.h>

int main(void) {
  write(fileno(stdout), "Hello world!\n", 13);
  return 0;
}

Cheers,
--
Igor
From alexbestms at math.uni-muenster.de  Tue Jun 30 18:21:06 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Tue Jun 30 18:21:13 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <20090630181121.GA32665@keira.kiwi-computer.com>
Message-ID: <permail-20090630182103f7e55a9d0000355c-a_best01@message-id.uni-muenster.de>

thanks. now the output gets redirected using >. i'm quite new to programming
under unix. sorry for the inconvenience.

so i guess there is no really easy way to output an inhomogeneous struct to
stdout without using a loop to output each array contained in the struct.

cheers.

Rick C. Petty schrieb am 2009-06-30:
> On Tue, Jun 30, 2009 at 08:03:21PM +0200, Alexander Best wrote:
> > should be stdout.


> > struct Header *hdr = rom;

> > int new_fd = open("/dev/stdout", O_RDWR);
> > printf("SIZE: %d\n",sizeof(*hdr));
> > write(new_fd, hdr, sizeof(*hdr));
> > close(new_fd);

> Why are you reopening stdout?  It should already be open, so use
> "fileno(stdout)" or just plain "STDOUT_FILENO" instead of "new_fd".

> -- Rick C. Petty
From stephen at missouri.edu  Tue Jun 30 19:27:50 2009
From: stephen at missouri.edu (Stephen Montgomery-Smith)
Date: Tue Jun 30 19:27:58 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-20090630182103f7e55a9d0000355c-a_best01@message-id.uni-muenster.de>
References: <permail-20090630182103f7e55a9d0000355c-a_best01@message-id.uni-muenster.de>
Message-ID: <4A4A64CE.1030906@missouri.edu>

Alexander Best wrote:
> thanks. now the output gets redirected using >. i'm quite new to programming
> under unix. sorry for the inconvenience.
> 
> so i guess there is no really easy way to output an inhomogeneous struct to
> stdout without using a loop to output each array contained in the struct.
> 
> cheers.

I think you are asking for a print command, that when given a structure, 
can look at the data type of each element of the structure, and produces 
the appropriate print command for each data type.

I think what you are asking for is antithetical to the philosophy of C - 
for example, the printf command demands that you tell the printf command 
what are the data types of the data you give to it.

So I think that the kind of long winded print command you are advocating 
is the correct way to do it.

Stephen
From rick-freebsd2008 at kiwi-computer.com  Tue Jun 30 21:02:39 2009
From: rick-freebsd2008 at kiwi-computer.com (Rick C. Petty)
Date: Tue Jun 30 21:02:46 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-20090630182103f7e55a9d0000355c-a_best01@message-id.uni-muenster.de>
References: <20090630181121.GA32665@keira.kiwi-computer.com>
	<permail-20090630182103f7e55a9d0000355c-a_best01@message-id.uni-muenster.de>
Message-ID: <20090630210238.GA33849@keira.kiwi-computer.com>

On Tue, Jun 30, 2009 at 08:21:03PM +0200, Alexander Best wrote:
> thanks. now the output gets redirected using >. i'm quite new to programming
> under unix. sorry for the inconvenience.

No problem; we all had to learn sometime.  But what I suggested should
work for every platform that adheres to POSIX.  If you were using
fprintf/fwrite, then it would work on anything that's standard C.  As for
redirection, windows command line allows the same type of redirection.

> so i guess there is no really easy way to output an inhomogeneous struct to
> stdout without using a loop to output each array contained in the struct.

That's not something C would ever provide easily.  You may want to use a
different high-level language.  However, I often use macros for printing
pieces of structures, for example I used this to print out sizes of kernel
structures:

#define SIZE(astruct, member) \
	printf("%d\t\t.%s\n", sizeof(astruct.member), #member)

#include <sys/ktrace.h>
...
	struct ktr_header header;
	struct ktr_genio genio;

	printf("%d\tktr_header:\n", sizeof(header));
	SIZE(header, ktr_len);
	SIZE(header, ktr_type);
	SIZE(header, ktr_pid);
	SIZE(header, ktr_comm);
	SIZE(header, ktr_time);
	SIZE(header, ktr_time.tv_sec);
	SIZE(header, ktr_time.tv_sec);
	SIZE(header, ktr_tid);

	printf("\n%d\tktr_genio:\n", sizeof(genio));
	SIZE(genio, ktr_fd);
	SIZE(genio, ktr_rw);

In your case, you could make a macro for each type.  Without an example of
how you want the output to look, it's hard for us to show you code that
will produce such output.

-- Rick C. Petty
From alexbestms at math.uni-muenster.de  Tue Jun 30 22:06:08 2009
From: alexbestms at math.uni-muenster.de (Alexander Best)
Date: Tue Jun 30 22:06:15 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <20090630210238.GA33849@keira.kiwi-computer.com>
Message-ID: <permail-2009063022060580e26a0b00003f4a-a_best01@message-id.uni-muenster.de>

thanks for all the help. i decided to take the pill and coded all the fprintfs
by hand. here's the result. usually i'd stick to a higher level languag, but i
need C's inline assembly support:

    struct Header
    {
        u_int8_t rom_entry[4];
        u_int8_t nintendo_logo[156];
        u_char game_title[12];
        u_char game_code[4];
        u_char maker_code[2];
        u_int8_t fixed_val;
        u_int8_t unit_code;
        u_int8_t device_type;
        u_int8_t reserved_area1[7];
        u_int8_t software_version;
        u_int8_t complement_check;
        u_int8_t reserved_area2;
        u_int8_t ram_entry[4];
        u_int8_t boot_mode;
        u_int8_t slave_id;
        u_int8_t unused_area[26];
        u_int8_t joybus_entry[4];
    };

    struct Header * hdr = rom;
    int i;

    fprintf(stderr, "ROM Entry: 0x");
    for (i=0; i < 4; i++) fprintf(stderr, "%x", hdr->rom_entry[i]);
    fprintf(stderr, "\nNintendo Logo: 0x");
    for (i=0; i < sizeof(hdr->nintendo_logo); i++) fprintf(stderr, "%x",
    hdr->nintendo_logo[i]);
    fprintf(stderr, "\nGame Title: %s",  hdr->game_title);
    fprintf(stderr, "\nGame Code: %s",  hdr->game_code);
    fprintf(stderr, "\nMaker Code: %s",  hdr->maker_code);
    fprintf(stderr, "\nFixed Value: 0x");
    fprintf(stderr, "%x", hdr->fixed_val);
    fprintf(stderr, "\nUnit Code: 0x");
    fprintf(stderr, "%x", hdr->unit_code);
    fprintf(stderr, "\nDevice Type: 0x");
    fprintf(stderr, "%x", hdr->device_type);
    fprintf(stderr, "\nReserved Area: 0x");
    for (i=0; i < sizeof(hdr->reserved_area1); i++) fprintf(stderr, "%x",
    hdr->reserved_area1[i]);
    fprintf(stderr, "\nSoftware Version: 0x");
    fprintf(stderr, "%x", hdr->software_version);
    fprintf(stderr, "\nComplement Check: 0x");
    fprintf(stderr, "%x", hdr->complement_check);
    fprintf(stderr, "\nReserved Area: 0x");
    fprintf(stderr, "%x", hdr->reserved_area2);
    fprintf(stderr, "\nRAM Entry Point: 0x");
    for (i=0; i < sizeof(hdr->ram_entry); i++) fprintf(stderr, "%x",
    hdr->ram_entry[i]);
    fprintf(stderr, "\nBoot Mode: 0x");
    fprintf(stderr, "%x", hdr->boot_mode);
    fprintf(stderr, "\nSlave ID: 0x");
    fprintf(stderr, "%x", hdr->slave_id);
    fprintf(stderr, "\nUnused Area: 0x");
    for (i=0; i < sizeof(hdr->unused_area); i++) fprintf(stderr, "%x",
    hdr->unused_area[i]);
    fprintf(stderr, "\nJoybus Entry Point: 0x");
    for (i=0; i < sizeof(hdr->joybus_entry); i++) fprintf(stderr, "%x",
    hdr->joybus_entry[i]);

cheers.

Rick C. Petty schrieb am 2009-06-30:
> On Tue, Jun 30, 2009 at 08:21:03PM +0200, Alexander Best wrote:
> > thanks. now the output gets redirected using >. i'm quite new to
> > programming
> > under unix. sorry for the inconvenience.

> No problem; we all had to learn sometime.  But what I suggested
> should
> work for every platform that adheres to POSIX.  If you were using
> fprintf/fwrite, then it would work on anything that's standard C.  As
> for
> redirection, windows command line allows the same type of
> redirection.

> > so i guess there is no really easy way to output an inhomogeneous
> > struct to
> > stdout without using a loop to output each array contained in the
> > struct.

> That's not something C would ever provide easily.  You may want to
> use a
> different high-level language.  However, I often use macros for
> printing
> pieces of structures, for example I used this to print out sizes of
> kernel
> structures:

> #define SIZE(astruct, member) \
>         printf("%d\t\t.%s\n", sizeof(astruct.member), #member)

> #include <sys/ktrace.h>
> ...
>         struct ktr_header header;
>         struct ktr_genio genio;

>         printf("%d\tktr_header:\n", sizeof(header));
>         SIZE(header, ktr_len);
>         SIZE(header, ktr_type);
>         SIZE(header, ktr_pid);
>         SIZE(header, ktr_comm);
>         SIZE(header, ktr_time);
>         SIZE(header, ktr_time.tv_sec);
>         SIZE(header, ktr_time.tv_sec);
>         SIZE(header, ktr_tid);

>         printf("\n%d\tktr_genio:\n", sizeof(genio));
>         SIZE(genio, ktr_fd);
>         SIZE(genio, ktr_rw);

> In your case, you could make a macro for each type.  Without an
> example of
> how you want the output to look, it's hard for us to show you code
> that
> will produce such output.

> -- Rick C. Petty
From mel.flynn+fbsd.hackers at mailing.thruhere.net  Tue Jun 30 22:31:55 2009
From: mel.flynn+fbsd.hackers at mailing.thruhere.net (Mel Flynn)
Date: Tue Jun 30 22:32:03 2009
Subject: large pages (amd64)
In-Reply-To: <4A49CBA0.1050806@cs.rice.edu>
References: <alpine.BSF.2.00.0906281933580.1809@wojtek.tensor.gdynia.pl>
	<200906291317.12040.mel.flynn+fbsd.hackers@mailing.thruhere.net>
	<4A49CBA0.1050806@cs.rice.edu>
Message-ID: <200906301431.38664.mel.flynn+fbsd.hackers@mailing.thruhere.net>

On Tuesday 30 June 2009 00:24:00 Alan Cox wrote:
> Mel Flynn wrote:
> > On Sunday 28 June 2009 15:41:49 Alan Cox wrote:
> >> Wojciech Puchar wrote:
> >>> how can i check how much (or maybe - what processes) 2MB pages are
> >>> actually allocated?
> >>
> >> I'm afraid that you can't with great precision.  For a given program
> >> execution, on an otherwise idle machine, you can only estimate the
> >> number by looking at the change in the quantity "promotions + mappings -
> >> demotions" before, during, and after the program execution.
> >>
> >> A program can call mincore(2) in order to determine if a virtual address
> >> is part of a 2 or 4MB virtual page.
> >
> > Would it be possible to expose the super page count as kve_super in the
> > kinfo_vmentry struct so that procstat can show this information? If only
> > to determine if one is using the feature and possibly benefiting from it.
>
> Yes, I think so.
>
> > It looks like sys/kern/kern_proc.c could call mincore around the loop at
> > line 1601 (rev 194498), but I know nothing about the vm subsystem to know
> > the implications or locking involved. There's still 16 bytes of spare to
> > consume, in the kve_vminfo struct though ;)
>
> Yes, to start with, you could replace the call to pmap_extract() with a
> call to pmap_mincore() and export a Boolean to user space that says,
> "This region of the address space contains one or more superpage mappings."

How about attached?

% sudo procstat -av|grep 'S '
  PID      START        END PRT  RES PRES REF SHD  FL TP PATH
 1754 0x28900000 0x2ae00000 rw- 9385    0   3   0 --S df
 2141 0x2f900000 0x30800000 rw- 3719    0   1   0 --S df
 2146 0x3eec0000 0x4fac0000 rwx 1745    0   1   0 --S df

-- 
Mel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: procstat-superpages.patch
Type: text/x-patch
Size: 2368 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20090630/dd3647d2/procstat-superpages.bin
From alfred at freebsd.org  Tue Jun 30 22:54:57 2009
From: alfred at freebsd.org (Alfred Perlstein)
Date: Tue Jun 30 22:55:04 2009
Subject: c question: *printf'ing arrays
In-Reply-To: <permail-2009063022060580e26a0b00003f4a-a_best01@message-id.uni-muenster.de>
References: <20090630210238.GA33849@keira.kiwi-computer.com>
	<permail-2009063022060580e26a0b00003f4a-a_best01@message-id.uni-muenster.de>
Message-ID: <20090630225457.GR84786@elvis.mu.org>

Hey Alex,

People frown on macros, but this could be a good one:

#define SPRINT(f, fmt) \
	do {\
		for (_i = 0; _i < sizeof(f)/sizeof(f[0]); i++) \
			printf(fmt, f[i]); \
	}while(0)

:D

This should allow you to point to any _array_ and print each
element of it using format "fmt".

Example:
SPRINT(Header->game_title, "%c");



-Alfred
	

* Alexander Best <alexbestms@math.uni-muenster.de> [090630 15:06] wrote:
> thanks for all the help. i decided to take the pill and coded all the fprintfs
> by hand. here's the result. usually i'd stick to a higher level languag, but i
> need C's inline assembly support:
> 
>     struct Header
>     {
>         u_int8_t rom_entry[4];
>         u_int8_t nintendo_logo[156];
>         u_char game_title[12];
>         u_char game_code[4];
>         u_char maker_code[2];
>         u_int8_t fixed_val;
>         u_int8_t unit_code;
>         u_int8_t device_type;
>         u_int8_t reserved_area1[7];
>         u_int8_t software_version;
>         u_int8_t complement_check;
>         u_int8_t reserved_area2;
>         u_int8_t ram_entry[4];
>         u_int8_t boot_mode;
>         u_int8_t slave_id;
>         u_int8_t unused_area[26];
>         u_int8_t joybus_entry[4];
>     };
> 
>     struct Header * hdr = rom;
>     int i;
> 
>     fprintf(stderr, "ROM Entry: 0x");
>     for (i=0; i < 4; i++) fprintf(stderr, "%x", hdr->rom_entry[i]);
>     fprintf(stderr, "\nNintendo Logo: 0x");
>     for (i=0; i < sizeof(hdr->nintendo_logo); i++) fprintf(stderr, "%x",
>     hdr->nintendo_logo[i]);
>     fprintf(stderr, "\nGame Title: %s",  hdr->game_title);
>     fprintf(stderr, "\nGame Code: %s",  hdr->game_code);
>     fprintf(stderr, "\nMaker Code: %s",  hdr->maker_code);
>     fprintf(stderr, "\nFixed Value: 0x");
>     fprintf(stderr, "%x", hdr->fixed_val);
>     fprintf(stderr, "\nUnit Code: 0x");
>     fprintf(stderr, "%x", hdr->unit_code);
>     fprintf(stderr, "\nDevice Type: 0x");
>     fprintf(stderr, "%x", hdr->device_type);
>     fprintf(stderr, "\nReserved Area: 0x");
>     for (i=0; i < sizeof(hdr->reserved_area1); i++) fprintf(stderr, "%x",
>     hdr->reserved_area1[i]);
>     fprintf(stderr, "\nSoftware Version: 0x");
>     fprintf(stderr, "%x", hdr->software_version);
>     fprintf(stderr, "\nComplement Check: 0x");
>     fprintf(stderr, "%x", hdr->complement_check);
>     fprintf(stderr, "\nReserved Area: 0x");
>     fprintf(stderr, "%x", hdr->reserved_area2);
>     fprintf(stderr, "\nRAM Entry Point: 0x");
>     for (i=0; i < sizeof(hdr->ram_entry); i++) fprintf(stderr, "%x",
>     hdr->ram_entry[i]);
>     fprintf(stderr, "\nBoot Mode: 0x");
>     fprintf(stderr, "%x", hdr->boot_mode);
>     fprintf(stderr, "\nSlave ID: 0x");
>     fprintf(stderr, "%x", hdr->slave_id);
>     fprintf(stderr, "\nUnused Area: 0x");
>     for (i=0; i < sizeof(hdr->unused_area); i++) fprintf(stderr, "%x",
>     hdr->unused_area[i]);
>     fprintf(stderr, "\nJoybus Entry Point: 0x");
>     for (i=0; i < sizeof(hdr->joybus_entry); i++) fprintf(stderr, "%x",
>     hdr->joybus_entry[i]);
> 
> cheers.
> 
> Rick C. Petty schrieb am 2009-06-30:
> > On Tue, Jun 30, 2009 at 08:21:03PM +0200, Alexander Best wrote:
> > > thanks. now the output gets redirected using >. i'm quite new to
> > > programming
> > > under unix. sorry for the inconvenience.
> 
> > No problem; we all had to learn sometime.  But what I suggested
> > should
> > work for every platform that adheres to POSIX.  If you were using
> > fprintf/fwrite, then it would work on anything that's standard C.  As
> > for
> > redirection, windows command line allows the same type of
> > redirection.
> 
> > > so i guess there is no really easy way to output an inhomogeneous
> > > struct to
> > > stdout without using a loop to output each array contained in the
> > > struct.
> 
> > That's not something C would ever provide easily.  You may want to
> > use a
> > different high-level language.  However, I often use macros for
> > printing
> > pieces of structures, for example I used this to print out sizes of
> > kernel
> > structures:
> 
> > #define SIZE(astruct, member) \
> >         printf("%d\t\t.%s\n", sizeof(astruct.member), #member)
> 
> > #include <sys/ktrace.h>
> > ...
> >         struct ktr_header header;
> >         struct ktr_genio genio;
> 
> >         printf("%d\tktr_header:\n", sizeof(header));
> >         SIZE(header, ktr_len);
> >         SIZE(header, ktr_type);
> >         SIZE(header, ktr_pid);
> >         SIZE(header, ktr_comm);
> >         SIZE(header, ktr_time);
> >         SIZE(header, ktr_time.tv_sec);
> >         SIZE(header, ktr_time.tv_sec);
> >         SIZE(header, ktr_tid);
> 
> >         printf("\n%d\tktr_genio:\n", sizeof(genio));
> >         SIZE(genio, ktr_fd);
> >         SIZE(genio, ktr_rw);
> 
> > In your case, you could make a macro for each type.  Without an
> > example of
> > how you want the output to look, it's hard for us to show you code
> > that
> > will produce such output.
> 
> > -- Rick C. Petty
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

-- 
- Alfred Perlstein
VMOA #5191, 03 vmax, 92 gs500, ch250 - FreeBSD