From info at videobrokers.com Mon Feb 2 00:50:23 2009 From: info at videobrokers.com (Video Brokers) Date: Mon Feb 2 00:50:30 2009 Subject: Second hand video equipment for sale. Message-ID: [1]WWW.VIDEOBROKERS.COM WE BUY & SALE USED VIDEO/BROADCAST EQUIPMENTS! Hello, Please find bellow some details regarding the equipment we have for sale at the moment. Do not hesitate to get in touch with us if what you are looking for is not listed here, let us know as well what you have for sale. VISION MIXER's : Sony DVS2000, 10 SDI inputs @ 5.000 Euros Sony DVS2000, 16 SDI inputs @ 7.000 Euros Abekas A8150, SDI @ 3.000 Euros BTS DD20, specs on request @ 8.000 Euros BTS DD30, specs on request @ 10.000 Euros BTS DD35, specs on request @ 14.000 Euros GVG 1200, SDI @ 5.000 Euros GVG 4000, specs on request @ 4.000 Euros GVG Kayak DD1, full options @21.000 Euros GVG Kayak HD200 & HD300, specs on request @ contact us for a quote Sony DVS7250 & 7350, specs on request @ contact us for a quote VTR's : Panasonic AJ-D230H, 200 original drum hours @ 1.100 Euros Panasonic AJ-D650E, 2400 original drum hours @ 1.000 Euros Panasonic AJ-D650E, with SDI, new drum "0hrs" @ 2.500 Euros Panasonic AJ-D950E, 2500 original drum hours @ 6.000 Euros Panasonic AJ-D960E, 1000 drum hours @ 7.000 Euros Panasonic AJLT-75E, 1500 original drum hours @5.500 Euros Panasonic AJLT95, new drum "0hrs" @ 9.000 Euros Panasonic AJHD1200E, 1200 original drum hours @ 9.000 Euros Panasonic AJHD1400E, 280 original drum hours @ 16.500 Euros Sony UVW1200P, details on request @ from 400 Euros Sony UVW1600P, details on request @ from 700 Euros Sony UVW1800P, details on request @ from 1.000 Euros Sony PVW2600P, details on request @ from 800 Euros Sony PVW2650P, new drum "0hrs" @ 1.500 Euros Sony PVW2800P, serviced @ 2.000 Euros Sony BVW60P, details on request @ from 400 Euros Sony BVW65P, details on request @ from 400 Euros Sony BVW70P, details on request @ from 2.000 Euros Sony BVW70P with SDI, 800 drum hours @ 2.800 Euros Sony BVW75P, details on request @ from 2.000 Euros Sony BVWD75P, 310 drum hours @ 2.800 Euros Sony DSR1P, 59 original drum hours @ 1.900 Euros Sony DSR40P, 680 original drum hours @ 1.900 Euros Sony DSR45P, 800 original drum hours @ 2.500 Euros Sony DSR60P, 900 drum hours @ 600 Euros Sony DSR80P, with SDI, 2600 original drum hours @ 2.200 Euros Sony DSR85P, with SDI, new drum "0hrs" @ 3.500 Euros Sony DSR1500AP, with fire wire and YUV, 680 original drum hours @ 3.500 Euros Sony DSR1500AP, with SDI& fire wire, 400 original drum hours @ 4.000 Euros Sony DSR2000P, 2600 original drum hours @ 5.500 Euros Sony DSR2000P, new drum "hrs" @ 7.000 Euros Sony DNWA30P, 2000 original drum hours @ 1.500 Euros Sony DNWA65P, 3000 drum hours @ 1.800 Euros Sony DNWA75P, 3500 original drum hours @ 4.500 Euros Sony DNWA220P, 2500 drum hours @ 4.000 Euros Sony DNWA225P, 2200 drum hours @ 7.000 Euros Sony J1, betacam SP and SX player, low hours @ 1.200 Euros Sony J3A, 1200 original drum hours @ 3.900 Euros Sony DVW522P, low original drum hours @ 2.500 Euros Sony DVW510P S/N 11198, 3278 drum hours @ 4.500 Euros Sony DVWA510P, 2500 drum hours @ 6.000 Euros Sony DVW500P S/N 16243, 3587 drum hours @ 15.000 Euros Sony DVWA500P S/N 10520, fully serviced, new drum "0hrs" @ 19.000 Euros Sony DVWM2000P, 300 original drum hours @ 23.000 Euros Sony JH3, 150 original drum hours, with fire wire @ 13.000 Euros Sony HDWM2000P S/N 49298, EX-DEMO, 390 original drum hours @ 33.000 Euros Sony HDWM2000P, NEW IN THE BOX @ 36.000 Euros CAMERA's & CAMCORDER's : Sony HDC1500, complete camera chain, specs on request, 16 units available @ 50.000 Euros/ unit Thomson LDK8000, complete camera chain, specs on request, 6 units available @ make an offer Sony DXCD30P, camera head @ 1.600 Euros Sony DXCD35P, camera head @ 2.000 Euros Thomson TTV1657D (4/3-16:9) complete triax chain @ 16.000 Euros Thomson LDK23HS MKII complete chain @ 38.000 Euros Panasonic AJD800P, 1400 drum hours @ 1.600 Euros Panasonic AJD610WAE, 700 drum hours @ 3.600 Euros Panasonic AJ-SDC615E, 1300 original drum hours @ 4.500 Euros Panasonic AJD910WAE, low hours @ 5.000 Euros Panasonic AGHVX200, ex-demo, 0hrs @ 3.300 Euros Panasonic AJSDX900, with AJ-VF20WE, 2550 original drum hours @ 7.000 Euros Panasonic AJHDX900, with AJ-HVF21, 1150 original drum hours @13.000 Euros Panasonic HPX-2100E, ex-demo, 4 years warranty, with view finder and microphone @ 18.700 Euros Panasonic AJ-HDC27 (varicam), 800 original drum hours @ 15.000 Euros Sony BVWD600P S/N 40358, 233 drum hours @ 2.000 Euros Sony HVR-Z1E, around 700 original drum hours @ 2.000 Euros Sony HVR-Z5E, NEW @ 3.740 Euros Sony HVR-Z7E, NEW @ 4.680 Euros Sony PMW-EX3, NEW @ 6.870 Euros Sony PMW-700, NEW @ 21.840 Euros Sony DSRPD150P, battery charger @ 1.200 Euros Sony DSRPD170P, battery charger, wide angle converter @ 1.600 Euros Sony DSR370P S/N 42129, 383 original drum hours, including Canon YH19x6.7KRS @ 3.500 Euros Sony DSR400P S/N 43512, 50 original drum hours @ 4.000 Euros Sony DSR500WSP S/N 42925, 800 original drum hours @ 3.000 Euros Sony DSR570WSP S/N 46925, 1200 original drum hours @ 4.500 Euros Sony DSR450WSP S/N 42790, 550 original drum hours @ 7.500 Euros Sony DVW700P, details on request @ from 2.000 Euros Sony DVW707P, details on request @ from 2.500 Euros Sony DNW7P, details on request @ from 2.000 Euros Sony DNW9WSP, details on request @ from 4.000 Euros Sony DNW90WSP, details on request @ from 6.000 Euros Sony DVW709WSP S/N 40090, 1317 original drum hours @ 9.000 Euros Sony DVW709WSP S/N 40011, 399 original drum hours @ 10.000 Euros Sony DVW790WS S/N 40330, 55 drum hours @ 13.500 Euros Sony DVW790WSP S/N 41929, 516 drum hours @ 15.000 Euros Sony DVW790WSP S/N 42199, 601 original drum hours @ 17.000 Euros Sony PDW530P S/N 40538, 40 original lazer hours @ 14.000 Euros Sony PDW530P S/N 60394, 18 original lazer hours @ 15.000 Euros Sony HDW730S S/N 10192, 551 original drum hours, like new ! @ 16.000 Euros Sony HDW750P S/N 40113, 1513 original drum hours, with down converter @ 17.000 Euros Sony HDW750P S/N 40112, 1521 original drum hours, with down converter @ 17.000 Euros Sony HDW790P, 70 original drum hours @ 23.000 Euros Sony HDWF900/3 S/N 12474, 1519 original drum hours @ 20.000 Euros LENSES : Canon ZSD300 & FPD400, ex-demo @ 1.500 Euros Fujinon A16x9BERM @ 800 Euros Canon J15x8BIRS @ 1.500 Euros Fujinon A15x8BEVM @ 1.500 Euros Fujinon A8.5x5.5BEVM @ 3.000 Euros Canon YJ19x9KRS @ 1.000 Euros Canon YJ12x6.5IRS @ 3.000 Euros Canon J16x8BIRS @ 2.500 Euros Canon J8x6BKRS @ 1.800 Euros Canon J8x6BIRS @ 2.500 Euros Canon J9x5.2BIRS @ 5.000 Euros Canon J11x4.5BIRS @ 7.000 Euros Canon J22x7.6IAS @ 6.000 Euros Canon J33x15IRS with lens support + remotes @ 17.000 Euros Fujinon A10x4.8BEVM @ 6.000 Euros Fujinon A13x4.5BERD @ 8.000 Euros Fujinon HA13x4.5BERM, @ 12.000 Euros Fujinon HA13x4.5BERM, EX-DEMO @ 13.000 Euros Fujinon HA20x7.8BM10 @ 13.000 Euros Fujinon HA10x5BM10 @ 13.000 Euros Canon HJ11x4.7BIRSE, NEW @ 13.000 Euros Canon HJ21x7.8BIRSD @ 11.000 Euros Canon HJ22x7.6BIRSE, EX-DEMO @ 13.000 Euros Canon HJ22x7.6BIASE, NEW @ 14.000 Euros Fujinon HA42x , perfect condition @ contact us Canon J55super, with zoom and focus + lens support @ 19.000 Euros Canon PJ70 MK1 with zoom and focus + lens support @ 24.000 Euros Canon PJ70 MKII with zoom and focus + lens support @ 34.000 Euros Canon DIGI SUPER 86 XS with zoom and focus @ 78.000 Euros Fujinon HAe 10x10 M T1.8 @ 45.000 Euros Fujinon Super prime set @ 14.000 Euros, including : HAe F5-M10 T1.5 and HAe F8-M10 T1.5 and HAe F20-M10 T1.5 VARIOUS : Sony DME3000 @ 3.000 Euros Sony DME7000 @ 6.000 Euros Sony CA701 @ 2.000 Euros Sony RMM7G @ 700 Euros Sony CCUM5P @ 900 Euros Sony CCUM7P @ 1.000 Euros Sony CA537P @ 700 Euros Snell & Wilcox TBS24D @ 2.000 Euros Pinnacle DEKO 500 @ 3.000 Euros Pinnacle DEKO 2000 @ 9.000 Euros Sony DSC1024 @ 1.000 Euros Panasonic BTLH900E @ 2.350 Euros Panasonic BTLH1700E @ 1.480 Euros Sony BVF-VC10W @ 1.700 Euros Sony LMD 1420 @ 500 Euros Sony LMD 2020 @ 600 Euros Sony PVM9L2 @ 500 Euros Sony PVM20M2, ex-demo @ 600 Euros Sony PVM20M4 SDI @ 1.600 Euros Sony BVM2016P with SDI @ 1.500 Euros Sony BVM20G1E with SDI & BKM10R @ 4.000 Euros Sony BVMD20F1E with BKM11RR & BKM21D & BKM14L @ 6.000 Euros Sony BVM-F24E, HD/SDI, 4000 operation hours @ 19.000 Euros Sony RM450 @ 600 Euros Sony PVE500 @ 900 Euros Sony BVE9100 @ 1.500 Euros Sony BVE2000 @ 1.500 Euros Tektronix 1721/1731 @ 600 Euros Tektronix SPG110 @ 400 Euros Tektronix SPG271 @ 1.000 Euros Tektronix TSG111 @ 500 Euros Fora FA320, TBC @ 500 Euros Fora FA330, TBC @ 600 Euros Vinten Vision 10 @ 1.300 Euros Vinten Vision 11, carbon fiber @ 2.000 Euros Sachtler S18+ SBMLCF @ 3.900 Euros Tektronix WFM300 @ 800 Euros Tektronix WFM601A @ 2.500 Euros Tektronix WFM5000, ex-demo @ 4.250 Euros Sony DMXE2000 @ 1.500 Euros Sony DMXE3000 @ 2.500 Euros Sony PCM7030 @ 600 Euros Sony PCM7040 @ 1.200 Euros Yamaha 03D @ 800 Euros DK AUDIO PT5210, Vari Time Digital Sync Generator @ 2.000 Euros IDX video wireless system SDI, Model: WIVI @ 2.700 Euros AccuScene VF 1280S HD view finder with zebra Black & White, compatible for F23/F35/Genesis/RED @ 9.000 Euros FOUCUS FS-100, fire store HD multiformat DVCPRO HD, 100Go @ 400 Euros Snell & Wilcox IQ MODULAR with IQADBBG and 2x IQAVDA @ 2.000 Euros EVS XT2 6 channels SD, 5x 73GO, audio analo, AES, 2x PSU cool swap, open code, multicam LSM, super motion, FX split screen, network SDTI 1.5, protocol VDCP/DD35/ODETICS/LOUTH, protocol AVSP @ 72.000 Euros Video Brokers [2]www.videobrokers.com Alexandre Villegoureix Tel : +33 (0)6.09.84.13.86 Email : [3]alex@videobrokers.fr [4]If you wish not to receive anyfirther offer from us please follow this link References 1. http://url.videobrokers.com/id.asp?l=51090-4030682-388168-966-0 2. http://url.videobrokers.com/id.asp?l=51090-4030682-388168-966-0 3. mailto:alex@videobrokers.fr 4. http://url.videobrokers.com/id.asp?l=51089-4030682-388168-966-0&id=388168-966-4030682-7705ac45&res=fr From bugmaster at FreeBSD.org Mon Feb 2 03:06:49 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Feb 2 03:07:27 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200902021106.n12B6mpt094369@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From obrien at FreeBSD.org Mon Feb 2 14:24:25 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Mon Feb 2 14:24:36 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090130.093052.-2022808221.imp@bsdimp.com> References: <200901130653.n0D6rrNX092719@svn.freebsd.org> <20090130015518.GA20404@hades.panopticon> <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> Message-ID: <20090202220628.GA76833@dragon.NUXI.org> On Fri, Jan 30, 2009 at 09:30:52AM -0700, M. Warner Losh wrote: > OMG! There's too much make output now, and the recent changes broke > useful make features. All but one were eventually fixed. I just > fixed the -s breakage. I fail to see where 'make -Q -s' is not suitable to achieve the goals you mentioned for 'make -s'. I did make sure you could quiet things. I've tried to measure the "~10% performance improvement" in using 'make -s' that was the technical justification for r187921. I have been unable to achieve ~10%. The summary is: 'make -j16 -Q -s' saves 2.54% over 'make -j16' on local disk. 'make -s' saves 0.96% over 'make' on local disk. results on NFS are insignificant [ delta < 0.5% ]. I ran benchmarks on ref8-amd64 (a machine others have access to and can audit my results). I checked out svn+ssh://svn.freebsd.org/base/head@187920 and built make from that. Using that make I then ran buildworld's. The abbreviated results for local disk are: MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 -Q buildworld >outfile 2>&1 25m38.07s real 1h30m6.59s user 45m37.27s sys 25m15.23s real 1h30m1.67s user 45m28.50s sys 25m22.65s real 1h30m8.85s user 45m32.10s sys (1538.07 + 1515.23 + 1522.65)/3 = 1525.32 sec ave [10% improvement would reduce build by 2m33s] MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 -Q -s buildworld >outfile 2>&1 24m36.12s real 1h27m27.18s user 44m26.88s sys 24m34.77s real 1h27m27.82s user 44m24.56s sys 24m29.01s real 1h27m22.16s user 44m28.23s sys (1476.12 + 1474.77 + 1469.01)/3 = 1473.30 sec ave => 1473.30 / 1525.32 * 100 - 100 = -3.41% change in build time [compared with below] => 1473.30 / 1511.70 * 100 - 100 = -2.54% change in build time MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 buildworld >outfile 2>&1 25m3.95s real 1h27m40.02s user 45m19.12s sys 24m57.35s real 1h27m33.31s user 45m6.81s sys 25m33.81s real 1h27m39.68s user 44m50.02s sys (1503.95 + 1497.35 + 1533.81)/3 = 1511.70 sec ave => 1511.70 / 1525.32 * 100 - 100 = -.89% change in build time MAKE=make@r187921 /usr/bin/time -h $MAKE buildworld >outfile 2>&1 1h48m7.30s real 1h28m22.07s user 22m28.01s sys 1h48m0.94s real 1h28m35.60s user 22m7.59s sys 1h48m4.20s real 1h28m39.28s user 21m58.27s sys (6487.30 + 6480.94 + 6484.20)/3 = 6484.15 sec ave [10% improvement would reduce build by 10m48s] MAKE=make@r187921 /usr/bin/time -h $MAKE -s buildworld >outfile 2>&1 1h47m56.48s real 1h28m35.91s user 22m0.57s sys 1h47m55.65s real 1h28m40.66s user 21m56.70s sys 1h51m5.05s real 1h28m48.22s user 22m21.96s sys (6476.48 + 6475.65 + 6665.05)/3 = 6539.06 sec ave => 6539.06 / 6484.15 * 100 - 100 = .85% change in build time If we toss out the high value and use 6476.48 twice, ave = 6476.20 => 6476.20 / 6539.06 * 100 - 100 = -.96% change in build time The NFS results are: MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 buildworld >outfile 2>&1 35m33.59s real 1h33m20.08s user 52m1.04s sys 31m9.58s real 1h33m43.16s user 52m41.46s sys 31m18.94s real 1h33m40.45s user 52m41.58s sys (2133.59 + 1869.58 + 1878.94)/3 = 1960.70 sec ave MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 -Q -s buildworld >outfile 2>&1 31m42.62s real 1h33m28.64s user 52m4.32s sys 31m14.54s real 1h33m21.25s user 52m8.88s sys 31m17.26s real 1h33m22.48s user 52m10.93s sys (1902.62 + 1874.54 + 1877.26)/3 = 1884.81 sec ave => 1884.81 / 1960.70 * 100 - 100 = -3.87% change in build time To be fair as in the local disk case, redoing the above: (1878.94 + 1869.58 + 1878.94)/3 = 1875.82 sec ave (adjusted) => 1884.81 / 1875.82 * 100 - 100 = .48% change in build time MAKE=make@r187921 /usr/bin/time -h $MAKE buildworld >outfile 2>&1 2h21m30.74s real 1h30m28.41s user 29m8.64s sys 2h21m2.15s real 1h30m25.54s user 29m2.99s sys 2h21m0.24s real 1h30m25.93s user 29m3.85s sys (8490.74 + 8462.15 + 8460.24)/3 = 8471.04 sec ave MAKE=make@r187921 /usr/bin/time -h $MAKE -s buildworld >outfile 2>&1 2h20m48.71s real 1h30m26.02s user 29m7.66s sys 2h21m5.95s real 1h30m21.87s user 29m10.28s sys 2h21m7.70s real 1h30m29.85s user 29m4.29s sys (8448.71 + 8465.95 + 8467.7)/3 = 8460.79 sec ave => 8460.79 /8471.04 * 100 - 100 = -.12% change in build time Thus I think there isn't any real time savings with "-Q -s"/"-s". Granted I was not in full control of the machine to ensure there was no completing usage, but from what I can other users didn't adversely affect my results. Also, three samples is rather small; but I was working from very little data from the "~10% performance improvement". I also "benchmarked" log size to see how much an issue that is: /usr/bin/time -h make@r187921 -j16 buildworld 30692989 , 30076086 , 30039499 (29M) /usr/bin/time -h $MAKE -j16 -Q buildworld 27734095, 27169263, 27169297 (26M) /usr/bin/time -h $MAKE -j16 -s buildworld 1168471 , 1168346 , 1165518 (1.1M) /usr/bin/time -h make@r187921 -j16 -Q -s buildworld 925073 , 925072 , 925103 (903K) /usr/bin/time -h make@r187921 buildworld 27168710 , 27168709 , 27168710 (26M) /usr/bin/time -h make@r187921 -s buildworld 924919 , 924920 , 924919 (903K) I do not consider the 3M of additional output of 'make -j16' vs. 'make -j16 -Q' to be significant with today's disk sizes. -- -- David From obrien at FreeBSD.org Mon Feb 2 14:31:48 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Mon Feb 2 14:31:54 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090130.093052.-2022808221.imp@bsdimp.com> References: <200901130653.n0D6rrNX092719@svn.freebsd.org> <20090130015518.GA20404@hades.panopticon> <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> Message-ID: <20090202223141.GB76833@dragon.NUXI.org> On Fri, Jan 30, 2009 at 09:30:52AM -0700, M. Warner Losh wrote: > First, there was absolutely no reason to introduce -Q. -v is > otherwise unused and matches the 'opt-in' debugging that's present in > the rest of make and the build system and other utilities like cp and > mv which will tell you what they are doing only if asked. I disagree. If one aliases "cp='cp -v'" it is easy to disable with issuing "\cp". Make is more complicated and looks at environmental variables to get its options. In those cases it is often useful to have a "disable" option that can used on the command line to override the environment. I could dig up other commands in /usr/src where one command enables something and following option disables it. ('ls -l -C' as one example) > Second, the extra always on debug introduces a performance penalty. I've replied separately on this topic. I was unable to measure any real 'make buildworld' penalty for my change. [that would be 'make -j16 -Q' vs. 'make -j16'] I was also only able to get about 2.5% speedup in a -j16 build after disabling all the output I easily could [make -j16 -Q -s] -- -- David (obrien@FreeBSD.org) From imp at bsdimp.com Mon Feb 2 14:42:22 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Feb 2 14:42:28 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090202220628.GA76833@dragon.NUXI.org> References: <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> <20090202220628.GA76833@dragon.NUXI.org> Message-ID: <20090202.154213.387237162.imp@bsdimp.com> In message: <20090202220628.GA76833@dragon.NUXI.org> "David O'Brien" writes: : On Fri, Jan 30, 2009 at 09:30:52AM -0700, M. Warner Losh wrote: : > OMG! There's too much make output now, and the recent changes broke : > useful make features. All but one were eventually fixed. I just : > fixed the -s breakage. : : I fail to see where 'make -Q -s' is not suitable to achieve the goals : you mentioned for 'make -s'. I did make sure you could quiet things. Two points here. (1) Should this be opt-in or opt-out. Is the output really so useful that we can't live without it? I think it should be -v to get the output. This is referred to as 'a -Q world' vs 'a -v world' in other discussions, so I'll use that here. What is the use-case that shows it is a win to be opt-in? What does having it on always buy us that having to repeat a run of a build with special debugging args to debug a problem fail to achieve? (2) Since this output falls outside of posix, one must also interpret the posix standard for what different flags do more loosely. The output is clearly related to the commands run. Is this enough, in a -Q world, to suppress it? That's not an automatic 'no' or 'yes'. In a -v world, it doesn't matter: we don't need to suppress additional output. : I've tried to measure the "~10% performance improvement" in using : 'make -s' that was the technical justification for r187921. I have : been unable to achieve ~10%. The summary is: [[ benchmarks deleted showing about a 2-3% drop in performance for -s ]] I was careful to caveat that with some environments. The one I was in was NFS to a slow server on an otherwise fast machine. I think the output size arguments are weak at best since they don't take into account the semantic value added by the output. Sure, disks can handle it, but does it add value for people or scripts reading the output? However, I think that it is a moot point since (a) I've back out the change and (b) the output is currently wrong. As to (b): Consider the following Makefile: all: one two three four one two three four: @echo ${.TARGET} @echo ${.TARGET} @echo ${.TARGET} @echo ${.TARGET} Right now, we get useless output from these job markers: % make -j 3 all --- one --- --- two --- --- three --- one one one --- one --- one two two two --- two --- two three three three --- three --- three --- four --- four four four four Based on this, I'd suggest we turn them off until we can at least produce good results. I don't know if we want them on by default with good results, but we certainly want them off when they are generating bad results. Warner From linimon at lonesome.com Mon Feb 2 15:41:59 2009 From: linimon at lonesome.com (Mark Linimon) Date: Mon Feb 2 15:42:05 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090202223141.GB76833@dragon.NUXI.org> References: <200901130653.n0D6rrNX092719@svn.freebsd.org> <20090130015518.GA20404@hades.panopticon> <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> <20090202223141.GB76833@dragon.NUXI.org> Message-ID: <20090202232417.GB6767@soaustin.net> Why are you being so obstinate about this? Hundreds of thousands of package builds have been done with make(1) before your "fix". Now there are gratuitous differences in the build output. Why? This is an immense POLA violation -- which is something you're very quick to beat other people over the head with. The fact that it was done this way in the historical past is irrelevant. The fact that other OSes may or may not do it this way is irrelevant. The fact that FreeBSD has done it the way that it has for -- what, 10 years -- *is* relevant, and is the *only* thing that is relevant. There are plenty of other _real_ problems in FreeBSD (e.g. ancient PRs assigned to committers that are long-undealt-with). Rather than arguing over a point you've already lost, why not work on those instead? mcl From obrien at FreeBSD.org Mon Feb 2 16:23:51 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Mon Feb 2 16:23:56 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090202.154213.387237162.imp@bsdimp.com> References: <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> <20090202220628.GA76833@dragon.NUXI.org> <20090202.154213.387237162.imp@bsdimp.com> Message-ID: <20090203002344.GB14870@dragon.NUXI.org> On Mon, Feb 02, 2009 at 03:42:13PM -0700, M. Warner Losh wrote: > Right now, we get useless output from these job markers: > % make -j 3 all .. > --- one --- > --- two --- > --- three --- > one > one > one > --- one --- > one > two > two > two > --- two --- > two > three .. snip.. > Based on this, I'd suggest we turn them off until we can at least > produce good results. I don't know if we want them on by default with > good results, but we certainly want them off when they are generating > bad results. I would have expected a few days to track down the bug. -- -- David (obrien@FreeBSD.org) From obrien at FreeBSD.org Mon Feb 2 16:46:49 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Mon Feb 2 16:46:55 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090202.154213.387237162.imp@bsdimp.com> References: <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> <20090202220628.GA76833@dragon.NUXI.org> <20090202.154213.387237162.imp@bsdimp.com> Message-ID: <20090203004642.GC14870@dragon.NUXI.org> On Mon, Feb 02, 2009 at 03:42:13PM -0700, M. Warner Losh wrote: > Two points here. > (1) Should this be opt-in or opt-out. Is the output really so useful > that we can't live without it? I think it should be -v to get the > output. This is referred to as 'a -Q world' vs 'a -v world' in > other discussions, so I'll use that here. What is the use-case > that shows it is a win to be opt-in? What does having it on > always buy us that having to repeat a run of a build with special > debugging args to debug a problem fail to achieve? It can be hard to get users to repeat something as-is. Enabling the job markers by default gives the information needed to understand issues is present. Much like Robert Watson's desire for textdumps - you want enough information to understand the problem when it occurred. Its hard to get users to go back and get the needed information. Maybe I'm the only one that has found it hard to pin point the reason for a build break with 'make -j'; but I do know over the years we've told folks that report build breaks while using 'make -j' to do the build again with no -j - one reason being its hard to figure out what happened in the build. As if, the way to get support is ``make -j8 buildworld || make buildworld'' > : I've tried to measure the "~10% performance improvement" in using > : 'make -s' that was the technical justification for r187921. I have > : been unable to achieve ~10%. The summary is: > > [[ benchmarks deleted showing about a 2-3% drop in performance for -s ]] > I was careful to caveat that with some environments. The one I was > in was NFS to a slow server on an otherwise fast machine. Actually the commit message said "many environments", but I think we've both added more details and folks can investigate for their own environment. > I think the > output size arguments are weak at best since they don't take into > account the semantic value added by the output. Sure, disks can > handle it, but does it add value for people or scripts reading the > output? However, I think that it is a moot point since (a) I've back > out the change and (b) the output is currently wrong. Once (b) is fixed, I do think it adds value for people and some scripts reading the output. Scripts can unwind the output and present the complete part of the build for a particular target. -- -- David (obrien@FreeBSD.org) From imp at bsdimp.com Mon Feb 2 19:27:07 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Mon Feb 2 19:27:18 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090203002344.GB14870@dragon.NUXI.org> References: <20090202220628.GA76833@dragon.NUXI.org> <20090202.154213.387237162.imp@bsdimp.com> <20090203002344.GB14870@dragon.NUXI.org> Message-ID: <20090202.202720.1492586314.imp@bsdimp.com> In message: <20090203002344.GB14870@dragon.NUXI.org> "David O'Brien" writes: : On Mon, Feb 02, 2009 at 03:42:13PM -0700, M. Warner Losh wrote: : > Right now, we get useless output from these job markers: : > % make -j 3 all : .. : > --- one --- : > --- two --- : > --- three --- : > one : > one : > one : > --- one --- : > one : > two : > two : > two : > --- two --- : > two : > three : .. snip.. : > Based on this, I'd suggest we turn them off until we can at least : > produce good results. I don't know if we want them on by default with : > good results, but we certainly want them off when they are generating : > bad results. : : I would have expected a few days to track down the bug. I don't think so. It is time to put this experiment into the background. The change in defaults is clearly unpopular (14-0 in favor of the patches I posted, not counting your and my votes). Its benefits are unproven at this time. It clearly isn't ready for prime time and needs more testing and careful thought and study. While there may be some minor benefits from this change, if it worked, the overwhelming negative feelings it has generated suggest that the time is not ripe for it socially and that more time is needed to socialize this change within the project before the switch is pulled. If we turn it off now, then that doesn't prevent you from fixing the functionality. It protects the users and developers from faulty data produced by the patches. It also calms down the very inflamed "public opinion" of the developers and users that have cared enough about the patch to send me their votes. I also don't think a couple of days will really matter, but we are wasting way too much time on this issue and it is time to move on and get on with our lives. If you can get it working, and show concrete examples of how it can benefit the project and convince the developers it is a good idea, then we can revisit the issue. Just my two cents, of course. Warner P.S. If you haven't sent me your vote, please do. I'm especially interested to hear from people that like the current patches and would like to see them stay, when fixed. From gad at FreeBSD.org Mon Feb 2 20:06:51 2009 From: gad at FreeBSD.org (Garance A Drosehn) Date: Mon Feb 2 20:06:57 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090202.154213.387237162.imp@bsdimp.com> References: <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> <20090202220628.GA76833@dragon.NUXI.org> <20090202.154213.387237162.imp@bsdimp.com> Message-ID: At 3:42 PM -0700 2/2/09, M. Warner Losh wrote: >[...] However, I think that it is a moot point since > (a) I've backed out the change and > (b) the output is currently wrong. > >As to (b): Consider the following Makefile: > >all: one two three four > >one two three four: > @echo ${.TARGET} > @echo ${.TARGET} > @echo ${.TARGET} > @echo ${.TARGET} > >Right now, we get useless output from these job markers: > >% make -j 3 all >--- one --- >--- two --- >--- three --- >one >one >one >--- one --- >one >two >two >two >--- two --- >two >three >three >three >--- three --- >three >--- four --- >four >four >four >four > >Based on this, I'd suggest we turn them off until we can at least >produce good results. I don't know if we want them on by default >with good results, but we certainly want them off when they are >generating bad results. It looks like the extra output assumes that each target is producing just a single line of output. That's pretty much what I expected would happen, and if so then the extra output doesn't really help all that much when it comes to debugging 'make -j'. I've looked around my various machines, and much to my surprise I think I found some debugging-changes I made to 'make' back in 2003/2004. These changes helped me understand some problems we were having with 'crunchgen' vs 'make -j' at the time. Let me see if I can figure out what the changes are doing, update them to the latest version of 'make', and see if people find those to be interesting. I might not have the time to do that until this weekend, though. (and just to make things more interesting, it seems I've found multiple versions of my changes... Arg.) I do remember the changes needed some more work before they'd be generally-wonderful, which is why I didn't try to commit them at the time. -- Garance Alistair Drosehn = drosehn@rpi.edu Senior Systems Programmer or gad@FreeBSD.org Rensselaer Polytechnic Institute; Troy, NY; USA From phk at phk.freebsd.dk Mon Feb 2 23:40:13 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon Feb 2 23:40:25 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: Your message of "Mon, 02 Feb 2009 14:06:29 PST." <20090202220628.GA76833@dragon.NUXI.org> Message-ID: <9061.1233646810@critter.freebsd.dk> In message <20090202220628.GA76833@dragon.NUXI.org>, "David O'Brien" writes: David, I am disappointed. You of all people here should know better than making such a mess out of benchmarks. First of all, you don't bother to even calculate a standard deviation even though we have a neat tool that does that in the base system, it's called "ministat", try it. Second you totally bungle your data collection, by not eliminating cache-effects. It should be evident to anybody that when your numbers always follow the pattern "high, low, low", that another run is necessary (you need 3 for stddev) and the first one should be discarded. Poul-Henning >The abbreviated results for local disk are: > >MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 -Q buildworld >outfile 2>&1 > 25m38.07s real 1h30m6.59s user 45m37.27s sys > 25m15.23s real 1h30m1.67s user 45m28.50s sys > 25m22.65s real 1h30m8.85s user 45m32.10s sys > (1538.07 + 1515.23 + 1522.65)/3 = 1525.32 sec ave > [10% improvement would reduce build by 2m33s] Standard deviation: 11.65 seconds >MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 -Q -s buildworld >outfile 2>&1 > 24m36.12s real 1h27m27.18s user 44m26.88s sys > 24m34.77s real 1h27m27.82s user 44m24.56s sys > 24m29.01s real 1h27m22.16s user 44m28.23s sys > (1476.12 + 1474.77 + 1469.01)/3 = 1473.30 sec ave > => 1473.30 / 1525.32 * 100 - 100 = -3.41% change in build time standard deviation: 3.77 second Difference at 95.0% confidence -52.0167 +/- 19.6298 -3.41022% +/- 1.28694% (Student's t, pooled s = 8.6605) > [compared with below] > => 1473.30 / 1511.70 * 100 - 100 = -2.54% change in build time Difference at 95.0% confidence 38.4033 +/- 31.7193 2.60662% +/- 2.15294% (Student's t, pooled s = 13.9942) >MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 buildworld >outfile 2>&1 > 25m3.95s real 1h27m40.02s user 45m19.12s sys > 24m57.35s real 1h27m33.31s user 45m6.81s sys > 25m33.81s real 1h27m39.68s user 44m50.02s sys > (1503.95 + 1497.35 + 1533.81)/3 = 1511.70 sec ave > => 1511.70 / 1525.32 * 100 - 100 = -.89% change in build time standard deviation: 19.42 second >MAKE=make@r187921 /usr/bin/time -h $MAKE buildworld >outfile 2>&1 > 1h48m7.30s real 1h28m22.07s user 22m28.01s sys > 1h48m0.94s real 1h28m35.60s user 22m7.59s sys > 1h48m4.20s real 1h28m39.28s user 21m58.27s sys > (6487.30 + 6480.94 + 6484.20)/3 = 6484.15 sec ave > [10% improvement would reduce build by 10m48s] standard deviation 3.18 seconds >MAKE=make@r187921 /usr/bin/time -h $MAKE -s buildworld >outfile 2>&1 > 1h47m56.48s real 1h28m35.91s user 22m0.57s sys > 1h47m55.65s real 1h28m40.66s user 21m56.70s sys > 1h51m5.05s real 1h28m48.22s user 22m21.96s sys > (6476.48 + 6475.65 + 6665.05)/3 = 6539.06 sec ave N Min Max Median Avg Stddev x 3 6475.65 6665.05 6476.48 6539.06 109.11133 + 3 6480.94 6487.3 6484.2 6484.1467 3.1803354 No difference proven at 95.0% confidence > If we toss out the high value and use 6476.48 twice, ave = 6476.20 > => 6476.20 / 6539.06 * 100 - 100 = -.96% change in build time We don't. Statistics is not a television show. >The NFS results are: > >MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 buildworld >outfile 2>&1 > 35m33.59s real 1h33m20.08s user 52m1.04s sys > 31m9.58s real 1h33m43.16s user 52m41.46s sys > 31m18.94s real 1h33m40.45s user 52m41.58s sys > (2133.59 + 1869.58 + 1878.94)/3 = 1960.70 sec ave standard deviation 149.79 seconds >MAKE=make@r187921 /usr/bin/time -h $MAKE -j16 -Q -s buildworld >outfile 2>&1 > 31m42.62s real 1h33m28.64s user 52m4.32s sys > 31m14.54s real 1h33m21.25s user 52m8.88s sys > 31m17.26s real 1h33m22.48s user 52m10.93s sys > (1902.62 + 1874.54 + 1877.26)/3 = 1884.81 sec ave > => 1884.81 / 1960.70 * 100 - 100 = -3.87% change in build time standard deviation 15.48 seconds N Min Max Median Avg Stddev x 3 1869.58 2133.59 1878.94 1960.7033 149.79737 + 3 1874.54 1902.62 1877.26 1884.8067 15.486631 No difference proven at 95.0% confidence > To be fair as in the local disk case, redoing the above: > (1878.94 + 1869.58 + 1878.94)/3 = 1875.82 sec ave (adjusted) > => 1884.81 / 1875.82 * 100 - 100 = .48% change in build time That has nothing to do with "fair", that is fudging the numbers. What you should have done, was realize that you need to run four tests and throw the first one which primes the cache away. >MAKE=make@r187921 /usr/bin/time -h $MAKE buildworld >outfile 2>&1 > 2h21m30.74s real 1h30m28.41s user 29m8.64s sys > 2h21m2.15s real 1h30m25.54s user 29m2.99s sys > 2h21m0.24s real 1h30m25.93s user 29m3.85s sys > (8490.74 + 8462.15 + 8460.24)/3 = 8471.04 sec ave standard deviation 17.08 seconds >MAKE=make@r187921 /usr/bin/time -h $MAKE -s buildworld >outfile 2>&1 > 2h20m48.71s real 1h30m26.02s user 29m7.66s sys > 2h21m5.95s real 1h30m21.87s user 29m10.28s sys > 2h21m7.70s real 1h30m29.85s user 29m4.29s sys > (8448.71 + 8465.95 + 8467.7)/3 = 8460.79 sec ave standard deviation 10.49 seconds No difference proven at 95.0% confidence -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From jhb at freebsd.org Tue Feb 3 06:48:25 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Feb 3 06:48:38 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090202232417.GB6767@soaustin.net> References: <200901130653.n0D6rrNX092719@svn.freebsd.org> <20090202223141.GB76833@dragon.NUXI.org> <20090202232417.GB6767@soaustin.net> Message-ID: <200902030948.13445.jhb@freebsd.org> On Monday 02 February 2009 6:24:17 pm Mark Linimon wrote: > Why are you being so obstinate about this? > > Hundreds of thousands of package builds have been done with make(1) > before your "fix". Now there are gratuitous differences in the build > output. Why? > > This is an immense POLA violation -- which is something you're very quick > to beat other people over the head with. The fact that it was done this > way in the historical past is irrelevant. The fact that other OSes may > or may not do it this way is irrelevant. The fact that FreeBSD has done > it the way that it has for -- what, 10 years -- *is* relevant, and is > the *only* thing that is relevant. > > There are plenty of other _real_ problems in FreeBSD (e.g. ancient PRs > assigned to committers that are long-undealt-with). Rather than arguing > over a point you've already lost, why not work on those instead? Yes, back it out now. Period. The fact that Warner is doing more work than just backing the whole thing out is very generous and if he wants to do that that is fine, but I would just back this whole mess out. It is nothing short of a POLA disaster for all of our _FreeBSD_ users who have been used to the way make has worked on FreeBSD for the past decade. -- John Baldwin From jhb at freebsd.org Tue Feb 3 06:48:25 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Feb 3 06:48:39 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090202232417.GB6767@soaustin.net> References: <200901130653.n0D6rrNX092719@svn.freebsd.org> <20090202223141.GB76833@dragon.NUXI.org> <20090202232417.GB6767@soaustin.net> Message-ID: <200902030948.13445.jhb@freebsd.org> On Monday 02 February 2009 6:24:17 pm Mark Linimon wrote: > Why are you being so obstinate about this? > > Hundreds of thousands of package builds have been done with make(1) > before your "fix". Now there are gratuitous differences in the build > output. Why? > > This is an immense POLA violation -- which is something you're very quick > to beat other people over the head with. The fact that it was done this > way in the historical past is irrelevant. The fact that other OSes may > or may not do it this way is irrelevant. The fact that FreeBSD has done > it the way that it has for -- what, 10 years -- *is* relevant, and is > the *only* thing that is relevant. > > There are plenty of other _real_ problems in FreeBSD (e.g. ancient PRs > assigned to committers that are long-undealt-with). Rather than arguing > over a point you've already lost, why not work on those instead? Yes, back it out now. Period. The fact that Warner is doing more work than just backing the whole thing out is very generous and if he wants to do that that is fine, but I would just back this whole mess out. It is nothing short of a POLA disaster for all of our _FreeBSD_ users who have been used to the way make has worked on FreeBSD for the past decade. -- John Baldwin From obrien at FreeBSD.org Tue Feb 3 07:40:24 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Tue Feb 3 07:40:31 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <9061.1233646810@critter.freebsd.dk> References: <20090202220628.GA76833@dragon.NUXI.org> <9061.1233646810@critter.freebsd.dk> Message-ID: <20090203154013.GA33520@dragon.NUXI.org> On Tue, Feb 03, 2009 at 07:40:10AM +0000, Poul-Henning Kamp wrote: > In message <20090202220628.GA76833@dragon.NUXI.org>, "David O'Brien" writes: > I am disappointed. > You of all people here should know better than making such a mess > out of benchmarks. Poul-Henning, I fully know these results are not stringent. Warner made a baseless 10% performance claim and used it as the bases for a commit. It took rounds of emails to get any detail from him, none of which were the actual times, standard deviation, etc... The only purpose of my measurements were to see if I could reproduce anything close to the claimed 10%. > Second you totally bungle your data collection, by not eliminating > cache-effects. I would have accepted Warner's 10% if I had gotten that just once, regardless if it was due to cache efforts or not. It would have backed up that, without totally cooking the runs, you could see a 10% time difference. -- David From obrien at FreeBSD.org Tue Feb 3 07:54:02 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Tue Feb 3 07:54:16 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: References: <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> <20090202220628.GA76833@dragon.NUXI.org> <20090202.154213.387237162.imp@bsdimp.com> Message-ID: <20090203155354.GC33520@dragon.NUXI.org> On Mon, Feb 02, 2009 at 11:06:40PM -0500, Garance A Drosehn wrote: > It looks like the extra output assumes that each target is > producing just a single line of output. That's pretty much what > I expected would happen, and if so then the extra output doesn't > really help all that much when it comes to debugging 'make -j'. No, there's a bug in our make. NetBSD's make gives the correct output. -- -- David (obrien@FreeBSD.org) From imp at bsdimp.com Tue Feb 3 07:57:53 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Tue Feb 3 07:58:00 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090203154013.GA33520@dragon.NUXI.org> References: <20090202220628.GA76833@dragon.NUXI.org> <9061.1233646810@critter.freebsd.dk> <20090203154013.GA33520@dragon.NUXI.org> Message-ID: <20090203.085631.-2006820702.imp@bsdimp.com> In message: <20090203154013.GA33520@dragon.NUXI.org> "David O'Brien" writes: : Warner made a baseless 10% performance claim and used it as the : bases for a commit. The claim was not baseless. It merely reflects results that were old. Warner From imp at bsdimp.com Tue Feb 3 08:12:45 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Tue Feb 3 08:12:53 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090203155354.GC33520@dragon.NUXI.org> References: <20090202.154213.387237162.imp@bsdimp.com> <20090203155354.GC33520@dragon.NUXI.org> Message-ID: <20090203.090959.-1377169487.imp@bsdimp.com> In message: <20090203155354.GC33520@dragon.NUXI.org> "David O'Brien" writes: : On Mon, Feb 02, 2009 at 11:06:40PM -0500, Garance A Drosehn wrote: : > It looks like the extra output assumes that each target is : > producing just a single line of output. That's pretty much what : > I expected would happen, and if so then the extra output doesn't : > really help all that much when it comes to debugging 'make -j'. : : No, there's a bug in our make. NetBSD's make gives the correct : output. make -s -v gives the correct output, but we don't have the right output w/o -s. That's a big clue there I think... Warner From obrien at FreeBSD.org Tue Feb 3 08:23:17 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Tue Feb 3 08:23:24 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090203.085631.-2006820702.imp@bsdimp.com> References: <20090202220628.GA76833@dragon.NUXI.org> <9061.1233646810@critter.freebsd.dk> <20090203154013.GA33520@dragon.NUXI.org> <20090203.085631.-2006820702.imp@bsdimp.com> Message-ID: <20090203162309.GD33520@dragon.NUXI.org> On Tue, Feb 03, 2009 at 08:56:31AM -0700, M. Warner Losh wrote: > In message: <20090203154013.GA33520@dragon.NUXI.org> > "David O'Brien" writes: > : Warner made a baseless 10% performance claim and used it as the > : bases for a commit. > > The claim was not baseless. It merely reflects results that were > old. Where were the runtime numbers? The standard deviation? Ministat output? Details of the machine run on? Information on the storage. You finally mentioned this was on FreeBSD 4.x, 5.2, and 6.0 build machines. A lot has changed since November 2005. And you were timing non-j build's - yet your change was all about parallel builds, it only affected 'make -j'. -- -- David (obrien@FreeBSD.org) From imp at bsdimp.com Tue Feb 3 08:30:26 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Tue Feb 3 08:30:32 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090203162309.GD33520@dragon.NUXI.org> References: <20090203154013.GA33520@dragon.NUXI.org> <20090203.085631.-2006820702.imp@bsdimp.com> <20090203162309.GD33520@dragon.NUXI.org> Message-ID: <20090203.092834.1062687916.imp@bsdimp.com> In message: <20090203162309.GD33520@dragon.NUXI.org> "David O'Brien" writes: : On Tue, Feb 03, 2009 at 08:56:31AM -0700, M. Warner Losh wrote: : > In message: <20090203154013.GA33520@dragon.NUXI.org> : > "David O'Brien" writes: : > : Warner made a baseless 10% performance claim and used it as the : > : bases for a commit. : > : > The claim was not baseless. It merely reflects results that were : > old. : : Where were the runtime numbers? The standard deviation? Ministat : output? Details of the machine run on? Information on the storage. : : You finally mentioned this was on FreeBSD 4.x, 5.2, and 6.0 build : machines. A lot has changed since November 2005. : : And you were timing non-j build's - yet your change was all about : parallel builds, it only affected 'make -j'. I'm done. Warner From julian at elischer.org Tue Feb 3 10:24:04 2009 From: julian at elischer.org (Julian Elischer) Date: Tue Feb 3 10:24:10 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090203162309.GD33520@dragon.NUXI.org> References: <20090202220628.GA76833@dragon.NUXI.org> <9061.1233646810@critter.freebsd.dk> <20090203154013.GA33520@dragon.NUXI.org> <20090203.085631.-2006820702.imp@bsdimp.com> <20090203162309.GD33520@dragon.NUXI.org> Message-ID: <49888BC9.4050704@elischer.org> David O'Brien wrote:I'm no tgoing to weigh in the technical aspects. but the general rule is you back out when asked and talk later.. can we pleas just do this? From obrien at FreeBSD.org Tue Feb 3 10:58:38 2009 From: obrien at FreeBSD.org (David O'Brien) Date: Tue Feb 3 10:58:45 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <49888BC9.4050704@elischer.org> References: <20090202220628.GA76833@dragon.NUXI.org> <9061.1233646810@critter.freebsd.dk> <20090203154013.GA33520@dragon.NUXI.org> <20090203.085631.-2006820702.imp@bsdimp.com> <20090203162309.GD33520@dragon.NUXI.org> <49888BC9.4050704@elischer.org> Message-ID: <20090203185836.GA41334@dragon.NUXI.org> On Tue, Feb 03, 2009 at 10:24:09AM -0800, Julian Elischer wrote: > David O'Brien wrote:I'm no tgoing to weigh in the technical > aspects. but the general rule is you back out when asked and talk later.. > can we pleas just do this? Hi Julian, I backed out the behavior this morning. -- -- David (obrien@FreeBSD.org) From gad at FreeBSD.org Tue Feb 3 11:35:39 2009 From: gad at FreeBSD.org (Garance A Drosehn) Date: Tue Feb 3 11:35:45 2009 Subject: svn commit: r187132 - head/usr.bin/make In-Reply-To: <20090203155354.GC33520@dragon.NUXI.org> References: <20090130.085130.-4349483.imp@bsdimp.com> <20090130.093052.-2022808221.imp@bsdimp.com> <20090202220628.GA76833@dragon.NUXI.org> <20090202.154213.387237162.imp@bsdimp.com> <20090203155354.GC33520@dragon.NUXI.org> Message-ID: At 7:53 AM -0800 2/3/09, David O'Brien wrote: >On Mon, Feb 02, 2009, Garance A Drosehn wrote: > > It looks like the extra output assumes that each target is >> producing just a single line of output. That's pretty much what >> I expected would happen, and if so then the extra output doesn't >> really help all that much when it comes to debugging 'make -j'. > >No, there's a bug in our make. NetBSD's make gives the correct >output. Well, I'm thinking about an issue which (I believe) is separate from the bug, but I'll wait until that bug is fixed before I take another look at the update I wrote. Maybe I did more work than I needed to at the time. -- Garance Alistair Drosehn = drosehn@rpi.edu Senior Systems Programmer or gad@FreeBSD.org Rensselaer Polytechnic Institute; Troy, NY; USA From avg at icyb.net.ua Fri Feb 6 04:18:03 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 04:18:10 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <20090131.212608.-1522433384.imp@bsdimp.com> References: <200901260947.32870.jhb@freebsd.org> <20090131093130.GA17896@dragon.NUXI.org> <20090131.212608.-1522433384.imp@bsdimp.com> Message-ID: <498C2A70.2030909@icyb.net.ua> on 01/02/2009 06:26 M. Warner Losh said the following: ... > hint.sc.0.at="isa" > hint.sc.0.flags="0x100" ... > I'm too chicken to remove the hint.sc.0 hints, but maybe they can go. > I've just tried this (7.1/i386) as those were the last hints, it turned out to be not that scary :) I just didn't get the system console, i.e. there were no kernel messages during boot and no system console at VT0 afterwards. VT0 became and stayed completely blank/black after loader finished its job. But the system successfully went into multi-user mode, getty-s and X started normally. OTOH, with this hint present I see the following in verbose dmesg: ... sc: sc0 already exists; skipping it ... sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: sc (syscons terminal) A little bit puzzling. And from the code in sys/isa/syscons_isa.c (scidentify) it seems that sc needs not a hint to get attached. Puzzled again. Let me try again with verbose dmesg that I can check once in multi-user. -- Andriy Gapon From rizzo at iet.unipi.it Fri Feb 6 06:57:07 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Fri Feb 6 06:57:14 2009 Subject: RFC: upcoming fixes to bioq_disksort() and friends. Message-ID: <20090206150252.GB11500@onelab2.iet.unipi.it> Hi, [people in Bcc were involved in a previous thread] we recently discovered that bioq_disksort() does not behave as intended, and the consensus on -developers was to revert the behaviour to the one in subr_disklabel.c v.1.1. Given that there are some corner cases that neither the original nor the current code address, we would like to describe how we are going to update the code. Please let us know if you have objections or there is anything unclear. The main methods for manipulating the bioq are bioq_disksort(), which does an ordered insertion, and bioq_first()/bioq_takefirst() which poll or extract the next request from the queue. This was the original API, and the behaviour of the bioq in this case is well specified. However, subr_disk.c also introduced and exported functions to directly manipulate the TAILQ embedded in the bioq. The behaviour of these functions in terms of keeping track of the disk head is not specified, and as a consequence, if you intermix bioq_disksort() and bioq_first()/bioq_takefirst() with the other calls, the ordering of the bioq could be completely disrupted [NOTE 1]. We are going to specify (and implement) the behaviour of the bioq as follows: 1. a bioq defines the current head position, last_offset, as the bio_offset of the first byte after [NOTE 2] the last entry extracted via bioq_takefirst(). 2. bioq_first() returns the next request with bio_offset >= last_offset, leaving the request in the queue; 3. bioq_takefirst() returns the next request with bio_offset >= last_offset, extracting it from the queue and setting last_offset according to #1 4. bioq_disksort() sorts requests using (uoff_t)(bio_offset - last_offset) as the sorting key (the offset space wraps); 5. bioq_insert_tail() appends to the end of the queue and DOES NOT change last_offset; 6. bioq_insert_head() inserts at the head of the queue and sets last_offset = bio_offset [NOTE 3]; 7. in terms of implementation, the field insert_point in struct bioq_head now becomes useless. However we are not going to remove it, so that parts of the kernel that #include bio.h do not need to be rebuilt. The change is only in the behaviour of the functions, and should be considered as a fix rather than a change of features. NOTES: 1. There are only two files that do this, sys/geom/mirror/g_mirror.c and sys/dev/xen/blkfront/blkfront.c. The g_vinum code also has some mix of code but it seems just a result of implementing two different queues with a single bioq, something that should be fixed anyways. 2. Historical behaviour is to just set last_offset = bio_offset. Accounting for the request size makes the code track the head position slightly better (assuming, of course, that we can make assumptions at all on the head position). But this makes a difference only in some corner cases. 3. The update of last_offset guarantees that a bioq_disksort() after a bioq_insert_head() puts the new entry after the one inserted bioq_insert_head(). We can leave last_offset untouched, in which case we lose that guarantee. Given that there is no precedent behaviour to preserve, both approaches are equally good, but we would prefer the one proposed (updating last_offset) as it makes it easier to reason on the behaviour of the queue. cheers luigi and fabio From jhb at freebsd.org Fri Feb 6 06:59:04 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 6 06:59:10 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <498C2A70.2030909@icyb.net.ua> References: <200901260947.32870.jhb@freebsd.org> <20090131.212608.-1522433384.imp@bsdimp.com> <498C2A70.2030909@icyb.net.ua> Message-ID: <200902060937.55724.jhb@freebsd.org> On Friday 06 February 2009 7:17:52 am Andriy Gapon wrote: > on 01/02/2009 06:26 M. Warner Losh said the following: > ... > > hint.sc.0.at="isa" > > hint.sc.0.flags="0x100" > ... > > I'm too chicken to remove the hint.sc.0 hints, but maybe they can go. > > > > I've just tried this (7.1/i386) as those were the last hints, it turned > out to be not that scary :) I just didn't get the system console, i.e. > there were no kernel messages during boot and no system console at VT0 > afterwards. VT0 became and stayed completely blank/black after loader > finished its job. > But the system successfully went into multi-user mode, getty-s and X > started normally. Yes, it's just the issue of the console that matters. If that could be made to work w/o needing the hint then we could remove it. > OTOH, with this hint present I see the following in verbose dmesg: > ... > sc: sc0 already exists; skipping it This is because it tried to add itself via an identify routine when it already existed. > ... > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > sc0: fb0, kbd1, terminal emulator: sc (syscons terminal) > > A little bit puzzling. > > And from the code in sys/isa/syscons_isa.c (scidentify) it seems that sc > needs not a hint to get attached. Puzzled again. Yes, it only needs the hint for it to be a console device. -- John Baldwin From avg at icyb.net.ua Fri Feb 6 07:22:40 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 07:22:45 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902060937.55724.jhb@freebsd.org> References: <200901260947.32870.jhb@freebsd.org> <20090131.212608.-1522433384.imp@bsdimp.com> <498C2A70.2030909@icyb.net.ua> <200902060937.55724.jhb@freebsd.org> Message-ID: <498C55BB.3030606@icyb.net.ua> on 06/02/2009 16:37 John Baldwin said the following: > > Yes, it only needs the hint for it to be a console device. > I am slightly confused as to how that hint works then, it's not like a standard isa hint it seems. Can it somehow be built-in (into the code)? -- Andriy Gapon From jhb at freebsd.org Fri Feb 6 08:07:07 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 6 08:07:14 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <498C55BB.3030606@icyb.net.ua> References: <200901260947.32870.jhb@freebsd.org> <200902060937.55724.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> Message-ID: <200902061106.41235.jhb@freebsd.org> On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: > on 06/02/2009 16:37 John Baldwin said the following: > > > > Yes, it only needs the hint for it to be a console device. > > > > I am slightly confused as to how that hint works then, it's not like a > standard isa hint it seems. > Can it somehow be built-in (into the code)? It works "normally" during the new-bus attach, but the low-level console code probably checks for it early on as well. The serial console code works that way. The uart(4) and sio(4) drivers explicitly look for uart/sio hints to find possible console devices. This is separate from when the ISA bus adds uart/sio devices from the hints later on. -- John Baldwin From imp at bsdimp.com Fri Feb 6 08:27:20 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri Feb 6 08:27:27 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061106.41235.jhb@freebsd.org> References: <200902060937.55724.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> <200902061106.41235.jhb@freebsd.org> Message-ID: <20090206.092523.2067072549.imp@bsdimp.com> In message: <200902061106.41235.jhb@freebsd.org> John Baldwin writes: : On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: : > on 06/02/2009 16:37 John Baldwin said the following: : > > : > > Yes, it only needs the hint for it to be a console device. : > > : > : > I am slightly confused as to how that hint works then, it's not like a : > standard isa hint it seems. : > Can it somehow be built-in (into the code)? : : It works "normally" during the new-bus attach, but the low-level console code : probably checks for it early on as well. The serial console code works that : way. The uart(4) and sio(4) drivers explicitly look for uart/sio hints to : find possible console devices. This is separate from when the ISA bus adds : uart/sio devices from the hints later on. sio could be loaded as a driver, and still be the console driver. But it had to be loaded from /boot/loader. Warner From jhb at freebsd.org Fri Feb 6 08:43:27 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 6 08:43:39 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <20090206.092523.2067072549.imp@bsdimp.com> References: <200902060937.55724.jhb@freebsd.org> <200902061106.41235.jhb@freebsd.org> <20090206.092523.2067072549.imp@bsdimp.com> Message-ID: <200902061137.42052.jhb@freebsd.org> On Friday 06 February 2009 11:25:23 am M. Warner Losh wrote: > In message: <200902061106.41235.jhb@freebsd.org> > John Baldwin writes: > : On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: > : > on 06/02/2009 16:37 John Baldwin said the following: > : > > > : > > Yes, it only needs the hint for it to be a console device. > : > > > : > > : > I am slightly confused as to how that hint works then, it's not like a > : > standard isa hint it seems. > : > Can it somehow be built-in (into the code)? > : > : It works "normally" during the new-bus attach, but the low-level console code > : probably checks for it early on as well. The serial console code works that > : way. The uart(4) and sio(4) drivers explicitly look for uart/sio hints to > : find possible console devices. This is separate from when the ISA bus adds > : uart/sio devices from the hints later on. > > sio could be loaded as a driver, and still be the console driver. But > it had to be loaded from /boot/loader. Are you sure? The console stuff is initialized (cninit()) well before any SYSINIT's are run. -- John Baldwin From jhb at freebsd.org Fri Feb 6 08:43:33 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 6 08:43:39 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <498C55BB.3030606@icyb.net.ua> References: <200901260947.32870.jhb@freebsd.org> <200902060937.55724.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> Message-ID: <200902061143.10088.jhb@freebsd.org> On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: > on 06/02/2009 16:37 John Baldwin said the following: > > > > Yes, it only needs the hint for it to be a console device. > > > > I am slightly confused as to how that hint works then, it's not like a > standard isa hint it seems. > Can it somehow be built-in (into the code)? Specifically, look at sc_cnprobe() in sys/dev/syscons/syscons.c. It calls a sc_cons_get_priority() routine that on x86 maps lives in sys/isa/syscons_isa.c. This checks for a syscons hint. Changing it to always assume a unit 0 would probably allow this to work. -- John Baldwin From avg at icyb.net.ua Fri Feb 6 08:48:05 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 08:48:12 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061106.41235.jhb@freebsd.org> References: <200901260947.32870.jhb@freebsd.org> <200902060937.55724.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> <200902061106.41235.jhb@freebsd.org> Message-ID: <498C69C1.70605@icyb.net.ua> on 06/02/2009 18:06 John Baldwin said the following: > On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: >> on 06/02/2009 16:37 John Baldwin said the following: >>> Yes, it only needs the hint for it to be a console device. >>> >> I am slightly confused as to how that hint works then, it's not like a >> standard isa hint it seems. >> Can it somehow be built-in (into the code)? > > It works "normally" during the new-bus attach, but the low-level console code > probably checks for it early on as well. The serial console code works that > way. The uart(4) and sio(4) drivers explicitly look for uart/sio hints to > find possible console devices. This is separate from when the ISA bus adds > uart/sio devices from the hints later on. > John, could it be sc_get_cons_priority function in syscons_isa.c? It seems that it explicitly searches through hints data and makes some important decisions based on it. It seems that without any hints at all it would return CN_DEAD. Maybe that code could be smarter and return something correct without relying that much on hints. It really doesn't seem like the hints should be so critical there (e.g. see ifdef XBOX there). -- Andriy Gapon From avg at icyb.net.ua Fri Feb 6 08:48:46 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 08:48:52 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061143.10088.jhb@freebsd.org> References: <200901260947.32870.jhb@freebsd.org> <200902060937.55724.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> <200902061143.10088.jhb@freebsd.org> Message-ID: <498C69EA.6030202@icyb.net.ua> on 06/02/2009 18:43 John Baldwin said the following: > On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: >> on 06/02/2009 16:37 John Baldwin said the following: >>> Yes, it only needs the hint for it to be a console device. >>> >> I am slightly confused as to how that hint works then, it's not like a >> standard isa hint it seems. >> Can it somehow be built-in (into the code)? > > Specifically, look at sc_cnprobe() in sys/dev/syscons/syscons.c. It calls a > sc_cons_get_priority() routine that on x86 maps lives in > sys/isa/syscons_isa.c. This checks for a syscons hint. Changing it to > always assume a unit 0 would probably allow this to work. > Mid-air collision detected :-) -- Andriy Gapon From jhb at freebsd.org Fri Feb 6 08:56:03 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 6 08:56:10 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061143.10088.jhb@freebsd.org> References: <200901260947.32870.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> <200902061143.10088.jhb@freebsd.org> Message-ID: <200902061155.48705.jhb@freebsd.org> On Friday 06 February 2009 11:43:09 am John Baldwin wrote: > On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: > > on 06/02/2009 16:37 John Baldwin said the following: > > > > > > Yes, it only needs the hint for it to be a console device. > > > > > > > I am slightly confused as to how that hint works then, it's not like a > > standard isa hint it seems. > > Can it somehow be built-in (into the code)? > > Specifically, look at sc_cnprobe() in sys/dev/syscons/syscons.c. It calls a > sc_cons_get_priority() routine that on x86 maps lives in > sys/isa/syscons_isa.c. This checks for a syscons hint. Changing it to > always assume a unit 0 would probably allow this to work. Something like this (untested): --- //depot/user/jhb/acpipci/isa/syscons_isa.c +++ /home/jhb/work/p4/acpipci/isa/syscons_isa.c @@ -238,8 +238,10 @@ *flags = f; } } - if (*unit < 0) - return CN_DEAD; + if (*unit < 0) { + *unit = 0; + *flags = 0; + } #if 0 return ((*flags & SC_KERNEL_CONSOLE) ? CN_INTERNAL : CN_NORMAL); #endif -- John Baldwin From avg at icyb.net.ua Fri Feb 6 09:05:17 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 09:05:24 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061155.48705.jhb@freebsd.org> References: <200901260947.32870.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> <200902061143.10088.jhb@freebsd.org> <200902061155.48705.jhb@freebsd.org> Message-ID: <498C6DC9.8020700@icyb.net.ua> on 06/02/2009 18:55 John Baldwin said the following: > On Friday 06 February 2009 11:43:09 am John Baldwin wrote: >> On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: >>> on 06/02/2009 16:37 John Baldwin said the following: >>>> Yes, it only needs the hint for it to be a console device. >>>> >>> I am slightly confused as to how that hint works then, it's not like a >>> standard isa hint it seems. >>> Can it somehow be built-in (into the code)? >> Specifically, look at sc_cnprobe() in sys/dev/syscons/syscons.c. It calls a >> sc_cons_get_priority() routine that on x86 maps lives in >> sys/isa/syscons_isa.c. This checks for a syscons hint. Changing it to >> always assume a unit 0 would probably allow this to work. > > Something like this (untested): I am not sure, but maybe, just in case, also add sc_get_softc(0,0) != NULL check? I guess device_get_softc returns NULL for non-attached/unknown devices. > --- //depot/user/jhb/acpipci/isa/syscons_isa.c > +++ /home/jhb/work/p4/acpipci/isa/syscons_isa.c > @@ -238,8 +238,10 @@ > *flags = f; > } > } > - if (*unit < 0) > - return CN_DEAD; > + if (*unit < 0) { > + *unit = 0; > + *flags = 0; > + } > #if 0 > return ((*flags & SC_KERNEL_CONSOLE) ? CN_INTERNAL : CN_NORMAL); > #endif > > -- Andriy Gapon From imp at bsdimp.com Fri Feb 6 09:07:19 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Fri Feb 6 09:07:25 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061137.42052.jhb@freebsd.org> References: <200902061106.41235.jhb@freebsd.org> <20090206.092523.2067072549.imp@bsdimp.com> <200902061137.42052.jhb@freebsd.org> Message-ID: <20090206.100535.-221723595.imp@bsdimp.com> In message: <200902061137.42052.jhb@freebsd.org> John Baldwin writes: : On Friday 06 February 2009 11:25:23 am M. Warner Losh wrote: : > In message: <200902061106.41235.jhb@freebsd.org> : > John Baldwin writes: : > : On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: : > : > on 06/02/2009 16:37 John Baldwin said the following: : > : > > : > : > > Yes, it only needs the hint for it to be a console device. : > : > > : > : > : > : > I am slightly confused as to how that hint works then, it's not like a : > : > standard isa hint it seems. : > : > Can it somehow be built-in (into the code)? : > : : > : It works "normally" during the new-bus attach, but the low-level console : code : > : probably checks for it early on as well. The serial console code works : that : > : way. The uart(4) and sio(4) drivers explicitly look for uart/sio hints to : > : find possible console devices. This is separate from when the ISA bus : adds : > : uart/sio devices from the hints later on. : > : > sio could be loaded as a driver, and still be the console driver. But : > it had to be loaded from /boot/loader. : : Are you sure? The console stuff is initialized (cninit()) well before any : SYSINIT's are run. I thought so, but maybe I'm just remembering that I could load it for my serial ports. I haven't had a laptop with serial ports on it in a while. Warner From avg at icyb.net.ua Fri Feb 6 09:14:46 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 09:14:53 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061155.48705.jhb@freebsd.org> References: <200901260947.32870.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> <200902061143.10088.jhb@freebsd.org> <200902061155.48705.jhb@freebsd.org> Message-ID: <498C7002.1090100@icyb.net.ua> Tangential: maybe AUTODETECT_KBD should be the default? I mean even without hints. I can not see any harm in it. -- Andriy Gapon From avg at icyb.net.ua Fri Feb 6 09:39:13 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 6 09:39:19 2009 Subject: NO_WERROR vs kernel builds In-Reply-To: <4989EA2A.6050601@icyb.net.ua> References: <4989EA2A.6050601@icyb.net.ua> Message-ID: <498C75BD.5040205@icyb.net.ua> on 04/02/2009 21:19 Andriy Gapon said the following: > It seems that kernel builds ignore NO_WERROR. > Is this on purpose or by accident? > > I think that this happens because of the following lines in > sys/conf/kern.pre.mk: > > .if ${CC} != "icc" > CFLAGS+= -fno-common -finline-limit=${INLINE_LIMIT} > CFLAGS+= --param inline-unit-growth=100 > CFLAGS+= --param large-function-growth=1000 > .if ${MACHINE_ARCH} == "amd64" || ${MACHINE} == "i386" || \ > ${MACHINE_ARCH} == "ia64" || ${MACHINE_ARCH} == "powerpc" || \ > ${MACHINE_ARCH} == "sparc64" > WERROR?= -Werror > .endif > .endif > > I had to specify WERROR= on make's command line to catch a certain kind > of warnings in bulk instead of one by one. This was not obvious. > Can anybody please explain or comment (or rub my nose into it)? -- Andriy Gapon From jhb at freebsd.org Fri Feb 6 10:02:19 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 6 10:02:31 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <498C6DC9.8020700@icyb.net.ua> References: <200901260947.32870.jhb@freebsd.org> <200902061155.48705.jhb@freebsd.org> <498C6DC9.8020700@icyb.net.ua> Message-ID: <200902061254.57192.jhb@freebsd.org> On Friday 06 February 2009 12:05:13 pm Andriy Gapon wrote: > on 06/02/2009 18:55 John Baldwin said the following: > > On Friday 06 February 2009 11:43:09 am John Baldwin wrote: > >> On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: > >>> on 06/02/2009 16:37 John Baldwin said the following: > >>>> Yes, it only needs the hint for it to be a console device. > >>>> > >>> I am slightly confused as to how that hint works then, it's not like a > >>> standard isa hint it seems. > >>> Can it somehow be built-in (into the code)? > >> Specifically, look at sc_cnprobe() in sys/dev/syscons/syscons.c. It calls a > >> sc_cons_get_priority() routine that on x86 maps lives in > >> sys/isa/syscons_isa.c. This checks for a syscons hint. Changing it to > >> always assume a unit 0 would probably allow this to work. > > > > Something like this (untested): > > I am not sure, but maybe, just in case, also add > sc_get_softc(0,0) != NULL > check? > I guess device_get_softc returns NULL for non-attached/unknown devices. This check happens long before new-bus gets started, so there are no softc's to check. -- John Baldwin From jhb at freebsd.org Fri Feb 6 10:02:24 2009 From: jhb at freebsd.org (John Baldwin) Date: Fri Feb 6 10:02:31 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <498C7002.1090100@icyb.net.ua> References: <200901260947.32870.jhb@freebsd.org> <200902061155.48705.jhb@freebsd.org> <498C7002.1090100@icyb.net.ua> Message-ID: <200902061255.12880.jhb@freebsd.org> On Friday 06 February 2009 12:14:42 pm Andriy Gapon wrote: > > Tangential: maybe AUTODETECT_KBD should be the default? > I mean even without hints. > I can not see any harm in it. Yes, it probably can just be changed so we don't need a hint to set it anymore. -- John Baldwin From ivoras at freebsd.org Fri Feb 6 19:12:24 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Fri Feb 6 19:12:37 2009 Subject: mount(8) in /stand? Message-ID: Judging by Google's results I'm only one of many people frustrated by the lack of mount(8) in the "emergency holographic shell". My problem is that I have everything I need to install the system (on a "netbook" laptop - no CD reader) on the USB drive I booted from, but no way to get to the data (the network drivers need to be patched before they can be used so net install is out, sysinstall doesn't recognize the directory structure, has no way of mounting msdosfs, etc.). I see the mount executable is ~~ 17 kB: -r-xr-xr-x 1 root wheel 17232 Dec 29 15:29 /sbin/mount* This is about the third time I needed it in similar circumstances so is probably not unreasonable to request it be crunched in for the future? It's certainly one of the basic emergency utilities. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 258 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090207/02cf8f9f/signature.pgp From keramida at ceid.upatras.gr Fri Feb 6 21:26:40 2009 From: keramida at ceid.upatras.gr (Giorgos Keramidas) Date: Fri Feb 6 21:26:47 2009 Subject: mount(8) in /stand? In-Reply-To: (Ivan Voras's message of "Sat, 07 Feb 2009 03:16:46 +0100") References: Message-ID: <87y6wivnmg.fsf@kobe.laptop> On Sat, 07 Feb 2009 03:16:46 +0100, Ivan Voras wrote: > Judging by Google's results I'm only one of many people frustrated by the > lack of mount(8) in the "emergency holographic shell". My problem is that > I have everything I need to install the system (on a "netbook" laptop - > no CD reader) on the USB drive I booted from, but no way to get to the > data (the network drivers need to be patched before they can be used so > net install is out, sysinstall doesn't recognize the directory structure, > has no way of mounting msdosfs, etc.). I see the mount executable is ~~ > 17 kB: > > -r-xr-xr-x 1 root wheel 17232 Dec 29 15:29 /sbin/mount* > > This is about the third time I needed it in similar circumstances so is > probably not unreasonable to request it be crunched in for the future? > It's certainly one of the basic emergency utilities. You have to account for the size of several mount_xxx executables too. My userland is now installed with DEBUG_FLAGS='-g' so the sizes are not as large as they seem below, but we need at least *some* of these to be in `/stand' before `/stand/mount' is usable e.g. for cd9660 mounts: % keramida@kobe:/sbin$ ls -ld mount* % -r-xr-xr-x 1 root wheel - 47382 Feb 6 23:33 mount % -r-xr-xr-x 1 root wheel - 23848 Feb 6 23:33 mount_cd9660 % -r-xr-xr-x 2 root wheel - 31741 Feb 6 23:33 mount_mfs % -r-xr-xr-x 1 root wheel - 27946 Feb 6 23:33 mount_msdosfs % -r-xr-xr-x 2 root wheel - 60590 Feb 6 23:33 mount_nfs % -r-xr-xr-x 2 root wheel - 60590 Feb 6 23:33 mount_nfs4 % -r-xr-xr-x 1 root wheel - 27835 Feb 6 23:33 mount_ntfs % -r-xr-xr-x 1 root wheel - 19567 Feb 6 23:33 mount_nullfs % -r-xr-xr-x 1 root wheel - 20836 Feb 6 23:33 mount_udf % -r-xr-xr-x 1 root wheel - 21853 Feb 6 23:33 mount_unionfs % keramida@kobe:/sbin$ Then, there's the obstacle of mount_nfs{,4} and all the symbols they will pull into the crunched binary. This may actually increase the disk space requirements of /stand a fair bit. With disks becoming cheaper every day, it may be sensible to include mount_xxx binaries in the default `/stand' for most of the installations, but provide WITHOUT_MOUNT_XXX knobs to allow striping of the parts that risk bloating /stand too much for smaller, embedded installations of BSD. From ivoras at freebsd.org Sat Feb 7 05:47:30 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Sat Feb 7 05:47:37 2009 Subject: mount(8) in /stand? In-Reply-To: <87y6wivnmg.fsf@kobe.laptop> References: <87y6wivnmg.fsf@kobe.laptop> Message-ID: <9bbcef730902070526q7c35a08dmfe98277d2029da7b@mail.gmail.com> 2009/2/7 Giorgos Keramidas : > On Sat, 07 Feb 2009 03:16:46 +0100, Ivan Voras wrote: >> Judging by Google's results I'm only one of many people frustrated by the >> lack of mount(8) in the "emergency holographic shell". My problem is that >> I have everything I need to install the system (on a "netbook" laptop - >> no CD reader) on the USB drive I booted from, but no way to get to the >> data (the network drivers need to be patched before they can be used so >> net install is out, sysinstall doesn't recognize the directory structure, >> has no way of mounting msdosfs, etc.). I see the mount executable is ~~ >> 17 kB: >> >> -r-xr-xr-x 1 root wheel 17232 Dec 29 15:29 /sbin/mount* >> >> This is about the third time I needed it in similar circumstances so is >> probably not unreasonable to request it be crunched in for the future? >> It's certainly one of the basic emergency utilities. > > You have to account for the size of several mount_xxx executables too. > My userland is now installed with DEBUG_FLAGS='-g' so the sizes are not as > large as they seem below, but we need at least *some* of these to be in > `/stand' before `/stand/mount' is usable e.g. for cd9660 mounts: What is the relationship between mount and mount_xxx? Is it that some file systems cannot be mounted at all if there's no mount_xxx or it's just there to provide advanced or unusual options? Actually what I'm asking is: can "local" file systems that normally have mount_xxx be mounted just with mount, assuming all options default, or they specifically and unconditionally require the special mount_xxx? I think msdosfs, cd9660 and udf are most important here. From rizzo at iet.unipi.it Sat Feb 7 07:45:31 2009 From: rizzo at iet.unipi.it (Luigi Rizzo) Date: Sat Feb 7 07:45:38 2009 Subject: RFC: upcoming fixes to bioq_disksort() and friends. Message-ID: <20090207155119.GC44298@onelab2.iet.unipi.it> As a followup, below is the core of the new bioq routines, as you can see they are extremely simple. An additional feature that might be useful, and trivial to implement, is let bioq_insert_tail() act as a barrier, so you have the guarantee that all new bioq_disksort() will be inserted after that. The necessart changes are in the #ifdef BARRIER/#endif blocks, and as you can see they are totally straightforward. If there are no objections, i am going to add them as well so the insert_tail() has a well-specified behaviour when intermixed with bioq_disksort() calls. cheers luigi ---- new implementation of bioq routines ------------- static inline uoff_t bioq_bio_key(struct bio_queue_head *head, struct bio *bp) { return ((uoff_t)(bp->bio_offset - head->last_offset)); } void bioq_disksort(struct bio_queue_head *head, struct bio *bp) { struct bio *cur, *prev = NULL; uoff_t key = bioq_bio_key(head, bp); cur = TAILQ_FIRST(&head->queue); #ifdef BARRIER if (head->insert_point) cur = head->insert_point; #endif /* BARRIER */ while (cur != NULL && key >= bioq_bio_key(head, cur)) { prev = cur; cur = TAILQ_NEXT(cur, bio_queue); } if (prev == NULL) TAILQ_INSERT_HEAD(&head->queue, bp, bio_queue); else TAILQ_INSERT_AFTER(&head->queue, prev, bp, bio_queue); } void bioq_remove(struct bio_queue_head *head, struct bio *bp) { if (bp == TAILQ_FIRST(&head->queue)) head->last_offset = bp->bio_offset + bp->bio_length; #ifdef BARRIER if (bp == head->insert_point) head->insert_point = NULL; #endif /* BARRIER */ TAILQ_REMOVE(&head->queue, bp, bio_queue); } struct bio * bioq_first(struct bio_queue_head *head) { return (TAILQ_FIRST(&head->queue)); } struct bio * bioq_takefirst(struct bio_queue_head *head) { struct bio *bp = TAILQ_FIRST(&head->queue); if (bp != NULL) bioq_remove(head, bp); return (bp); } void bioq_insert_head(struct bio_queue_head *head, struct bio *bp) { head->last_offset = bp->bio_offset; TAILQ_INSERT_HEAD(&head->queue, bp, bio_queue); } void bioq_insert_tail(struct bio_queue_head *head, struct bio *bp) { TAILQ_INSERT_TAIL(&head->queue, bp, bio_queue); #ifdef BARRIER head->insert_point = bp; #endif } void bioq_init(struct bio_queue_head *head) { TAILQ_INIT(&head->queue); head->last_offset = 0; head->insert_point = NULL; /* Unused or barrier */ } --------------------------------------------------------------------- From das at FreeBSD.ORG Sat Feb 7 11:10:18 2009 From: das at FreeBSD.ORG (David Schultz) Date: Sat Feb 7 11:10:24 2009 Subject: mount(8) in /stand? In-Reply-To: <9bbcef730902070526q7c35a08dmfe98277d2029da7b@mail.gmail.com> References: <87y6wivnmg.fsf@kobe.laptop> <9bbcef730902070526q7c35a08dmfe98277d2029da7b@mail.gmail.com> Message-ID: <20090207191011.GA7545@zim.MIT.EDU> On Sat, Feb 07, 2009, Ivan Voras wrote: > 2009/2/7 Giorgos Keramidas : > > On Sat, 07 Feb 2009 03:16:46 +0100, Ivan Voras wrote: > >> Judging by Google's results I'm only one of many people frustrated by the > >> lack of mount(8) in the "emergency holographic shell". My problem is that > >> I have everything I need to install the system (on a "netbook" laptop - > >> no CD reader) on the USB drive I booted from, but no way to get to the > >> data (the network drivers need to be patched before they can be used so > >> net install is out, sysinstall doesn't recognize the directory structure, > >> has no way of mounting msdosfs, etc.). I see the mount executable is ~~ > >> 17 kB: > >> > >> -r-xr-xr-x 1 root wheel 17232 Dec 29 15:29 /sbin/mount* > >> > >> This is about the third time I needed it in similar circumstances so is > >> probably not unreasonable to request it be crunched in for the future? > >> It's certainly one of the basic emergency utilities. > > > > You have to account for the size of several mount_xxx executables too. > > My userland is now installed with DEBUG_FLAGS='-g' so the sizes are not as > > large as they seem below, but we need at least *some* of these to be in > > `/stand' before `/stand/mount' is usable e.g. for cd9660 mounts: > > What is the relationship between mount and mount_xxx? Is it that some > file systems cannot be mounted at all if there's no mount_xxx or it's > just there to provide advanced or unusual options? Some filesystems have unusual options or require extra magic (e.g., loading kernel modules). These differences should be handled by the filesystem kernel code, but they're presently handled by having N nearly-identical mount_xxx binaries. Craig Rodrigues probably has a better understanding of what's still required to fix this. From imp at bsdimp.com Sat Feb 7 23:03:06 2009 From: imp at bsdimp.com (M. Warner Losh) Date: Sat Feb 7 23:03:12 2009 Subject: Small syntactic sugar addition to kobj Message-ID: <20090208.000115.-1145158523.imp@bsdimp.com> Greetings, I'd like to propose a small syntactic sugar coating for kobj. I'd like to propose: #define KOBJMETHOD_END { NULL, NULL } which can be used in place of the current { 0, 0 } used to end the lists in kobj function lists. Warner From gnn at freebsd.org Sun Feb 8 05:09:55 2009 From: gnn at freebsd.org (gnn@freebsd.org) Date: Sun Feb 8 05:10:01 2009 Subject: Small syntactic sugar addition to kobj In-Reply-To: <20090208.000115.-1145158523.imp@bsdimp.com> References: <20090208.000115.-1145158523.imp@bsdimp.com> Message-ID: At Sun, 08 Feb 2009 00:01:15 -0700 (MST), M. Warner Losh wrote: > > Greetings, > > I'd like to propose a small syntactic sugar coating for kobj. I'd > like to propose: > > #define KOBJMETHOD_END { NULL, NULL } > > which can be used in place of the current { 0, 0 } used to end the > lists in kobj function lists. That's an easy one. I don't see why we wouldn't do that. Later, George From received at postcard.org Sun Feb 8 20:37:43 2009 From: received at postcard.org (received@postcard.org) Date: Sun Feb 8 20:37:50 2009 Subject: You have just received a virtual postcard from a friend ! Message-ID: <20090209034400.3AFEC8DB285@mail.stett-online.de> You have just received a virtual postcard from a friend ! . You can pick up your postcard at the following web address: . [1]Click here to pick up your postcard . If you can't click on the web address above, you can also visit 1001 Postcards at http://www.postcards.org/postcards/ and enter your pickup code, which is: d21-sea-sunset . (Your postcard will be available for 60 days.) . Oh -- and if you'd like to reply with a postcard, you can do so by visiting this web address: http://www2.postcards.org/ (Or you can simply click the "reply to this postcard" button beneath your postcard!) . We hope you enjoy your postcard, and if you do, please take a moment to send a few yourself! . Regards, 1001 Postcards http://www.postcards.org/postcards/ References 1. http://hostevent.be/roundcube/bin/postcard.gif.exe From bugmaster at FreeBSD.org Mon Feb 9 03:06:49 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Feb 9 03:07:32 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200902091106.n19B6lCt009050@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From avg at icyb.net.ua Mon Feb 9 05:28:41 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Mon Feb 9 05:28:47 2009 Subject: Trimming the default /boot/device.hints In-Reply-To: <200902061155.48705.jhb@freebsd.org> References: <200901260947.32870.jhb@freebsd.org> <498C55BB.3030606@icyb.net.ua> <200902061143.10088.jhb@freebsd.org> <200902061155.48705.jhb@freebsd.org> Message-ID: <49902F7E.3080202@icyb.net.ua> on 06/02/2009 18:55 John Baldwin said the following: > On Friday 06 February 2009 11:43:09 am John Baldwin wrote: >> On Friday 06 February 2009 10:22:35 am Andriy Gapon wrote: >>> on 06/02/2009 16:37 John Baldwin said the following: >>>> Yes, it only needs the hint for it to be a console device. >>>> >>> I am slightly confused as to how that hint works then, it's not like a >>> standard isa hint it seems. >>> Can it somehow be built-in (into the code)? >> Specifically, look at sc_cnprobe() in sys/dev/syscons/syscons.c. It calls a >> sc_cons_get_priority() routine that on x86 maps lives in >> sys/isa/syscons_isa.c. This checks for a syscons hint. Changing it to >> always assume a unit 0 would probably allow this to work. > > Something like this (untested): > > --- //depot/user/jhb/acpipci/isa/syscons_isa.c > +++ /home/jhb/work/p4/acpipci/isa/syscons_isa.c > @@ -238,8 +238,10 @@ > *flags = f; > } > } > - if (*unit < 0) > - return CN_DEAD; > + if (*unit < 0) { > + *unit = 0; > + *flags = 0; > + } > #if 0 > return ((*flags & SC_KERNEL_CONSOLE) ? CN_INTERNAL : CN_NORMAL); > #endif > > Tested this patch - works great! - I am "hints free" now :-) This was stable/7, r188116, i386. -- Andriy Gapon From nwsadm at piekmarketing.eu Tue Feb 10 06:01:11 2009 From: nwsadm at piekmarketing.eu (Piek International Education Centre (I.E.C.)) Date: Tue Feb 10 06:01:23 2009 Subject: Newsletter PIEK International Education Center I.E.C. Message-ID: <6D3136DF241B4F6ABFD40F9CE13EF5DF@ZRE001> From nwsadm at piekmarketing.eu Tue Feb 10 06:29:57 2009 From: nwsadm at piekmarketing.eu (Piek International Education Centre (I.E.C.)) Date: Tue Feb 10 06:30:04 2009 Subject: Newsletter PIEK International Education Center I.E.C. Message-ID: <4E2903FA045B4AC895B686F1F10F835C@ZRE001> From ambrisko at ambrisko.com Thu Feb 12 11:28:12 2009 From: ambrisko at ambrisko.com (Doug Ambrisko) Date: Thu Feb 12 11:28:19 2009 Subject: rtld enhancement to add osversion sub-directory search Message-ID: <200902121859.n1CIxaJd083445@ambrisko.com> Hi folks, I'd like to discuss the idea of adding an automatic directory search feature to rtld. At work we need to run 3rd party code that we get as objects or even binaries that are compiled by other groups. This works okay when our base OS is the same as the 3rd party but not so good when we upgrade our base OS and we don't want to force th 3rd party code to upgrade at the same time. Sometimes they can't since legacy systems that needs to run the 3rd party code run the older OS. Now WRT to FreeBSD's base libraries this isn't much of a problem since the OS's lib's general bump version between releases so just via the name we have unique lib's and the loader can get the correct one. However, once you start to link to stuff in ports (ie. /usr/local/lib) then these names are no longer unique and something built for FreeBSD 4 through 7 (as part of FreeBSD's release, ie pre-built packages) end up with the same name. This is a problem since sticking the old version in /usr/lib/compat is a problem since it looks based on name (ie. /usr/local/lib/libiconv.so.3). Things can get interesting when the /usr/local/lib pulls in libc. So now your FreeBSD 4 binary could pull in a libiconv.so.3 for FreeBSD 7 and libc from FreeBSD 7. Interesting things start to happen! What I've done for work is to teach the rtld to look for the .note.ABI-tag and extract the osversion. I then put that and the osversion major number (ie. 6 instead of 603000) into a "try" list. Then whenever an object/lib is attemped to be accessed I look in the osversion sub-directory then the osversion major and finally the standard location. I even extended this to LD_PRELOAD and insert this try into the path just before the object/lib if fully qualified. Actually, I do it for all fully qualified things. Now I can stick most things into /usr/lib/compat/6 and it just works. For fully qualified LD_PRELOAD I "mv" things on boot into the sub-directory for that OS. Also I had to change LD_PRELOAD to accept failure if the thing wasn't found since children in herit that and if it a different version that LD_PRELOAD might not be applicable. I don't really see a maintain for rtld since jdp when I worked with him to add LD_PRELOAD. I'd like to get feed back on this idea and get it into FreeBSD. It would help solve some of these issues for other people as well. It definitely, made my life easier at work. It would also make things easier on my FreeBSD machines, when I upgrade the OS and ports but have legacy compiled things that I don't want to recompile again. I could move all of my /usr/local/libs into /usr/local/lib/compat/ References: <200902121859.n1CIxaJd083445@ambrisko.com> Message-ID: <20090212201101.GI2723@deviant.kiev.zoral.com.ua> On Thu, Feb 12, 2009 at 10:59:36AM -0800, Doug Ambrisko wrote: > Hi folks, > > I'd like to discuss the idea of adding an automatic directory search > feature to rtld. At work we need to run 3rd party code that we get > as objects or even binaries that are compiled by other groups. This > works okay when our base OS is the same as the 3rd party but not so > good when we upgrade our base OS and we don't want to force th 3rd > party code to upgrade at the same time. Sometimes they can't since > legacy systems that needs to run the 3rd party code run the older > OS. Now WRT to FreeBSD's base libraries this isn't much of a problem > since the OS's lib's general bump version between releases so just > via the name we have unique lib's and the loader can get the correct > one. However, once you start to link to stuff in ports (ie. > /usr/local/lib) then these names are no longer unique and something > built for FreeBSD 4 through 7 (as part of FreeBSD's release, ie > pre-built packages) end up with the same name. This is a problem since > sticking the old version in /usr/lib/compat is a problem since it > looks based on name (ie. /usr/local/lib/libiconv.so.3). Things can > get interesting when the /usr/local/lib pulls in libc. So now your > FreeBSD 4 binary could pull in a libiconv.so.3 for FreeBSD 7 and > libc from FreeBSD 7. Interesting things start to happen! > > What I've done for work is to teach the rtld to look for the > .note.ABI-tag and extract the osversion. I then put that and the > osversion major number (ie. 6 instead of 603000) into a "try" list. > Then whenever an object/lib is attemped to be accessed I look in the > osversion sub-directory then the osversion major and finally the > standard location. I even extended this to LD_PRELOAD and > insert this try into the path just before the object/lib if fully > qualified. Actually, I do it for all fully qualified things. > Now I can stick most things into /usr/lib/compat/6 and it just > works. For fully qualified LD_PRELOAD I "mv" things on boot > into the sub-directory for that OS. Also I had to change LD_PRELOAD > to accept failure if the thing wasn't found since children in herit > that and if it a different version that LD_PRELOAD might not be > applicable. > > I don't really see a maintain for rtld since jdp when I worked with > him to add LD_PRELOAD. I'd like to get feed back on this idea and > get it into FreeBSD. It would help solve some of these issues for > other people as well. It definitely, made my life easier at work. > It would also make things easier on my FreeBSD machines, when I > upgrade the OS and ports but have legacy compiled things that I > don't want to recompile again. I could move all of my /usr/local/libs > into /usr/local/lib/compat/ still run my old stuff. I've had issues before when OO was cross-linked > with libc from 6 and 7 due to this. Rebuilding OO was not fun and > I shouldn't have needed to. > > There is one glitch in atleast FreeBSD 6.1 didn't put the .note.ABI-tag > into binaries so for now if not specified I assume FreeBSD 6. That > is good enough for work but I could add an env. variable to hold > the default. If not set then it wouldn't do the sub-directory thing > on binaries that are not known. > > Loading of objects will slow done a bit since it adds extra > searches. It might be good enough to just do the os version major > and not for the complete osversion. > > BTW, I also added a feature to look at LD32_ first then LD_ > when running 32bit on 64bit. Legacy SW doesn't know about > LD32_ and just sets LD_. This way that stuff doesn't need > to be taught about LD32_ when it shouldn't really need to. There is a popular feature, unfortunately, not supported by FreeBSD ld.so, called Dynamic String Tokens, see http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view I have almost abandoned patch that adds support for $ORIGIN, $OSREL, $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT without serious conflicts. http://people.freebsd.org/~kib/misc/rtld_locks.4.patch -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090212/508a535f/attachment.pgp From ambrisko at ambrisko.com Thu Feb 12 13:19:26 2009 From: ambrisko at ambrisko.com (Doug Ambrisko) Date: Thu Feb 12 13:19:33 2009 Subject: rtld enhancement to add osversion sub-directory search In-Reply-To: <20090212201101.GI2723@deviant.kiev.zoral.com.ua> Message-ID: <200902122119.n1CLJOmI092041@ambrisko.com> Kostik Belousov writes: | On Thu, Feb 12, 2009 at 10:59:36AM -0800, Doug Ambrisko wrote: | > Hi folks, | > | > I'd like to discuss the idea of adding an automatic directory search | > feature to rtld. At work we need to run 3rd party code that we get | > as objects or even binaries that are compiled by other groups. This | > works okay when our base OS is the same as the 3rd party but not so | > good when we upgrade our base OS and we don't want to force th 3rd | > party code to upgrade at the same time. Sometimes they can't since | > legacy systems that needs to run the 3rd party code run the older | > OS. Now WRT to FreeBSD's base libraries this isn't much of a problem | > since the OS's lib's general bump version between releases so just | > via the name we have unique lib's and the loader can get the correct | > one. However, once you start to link to stuff in ports (ie. | > /usr/local/lib) then these names are no longer unique and something | > built for FreeBSD 4 through 7 (as part of FreeBSD's release, ie | > pre-built packages) end up with the same name. This is a problem since | > sticking the old version in /usr/lib/compat is a problem since it | > looks based on name (ie. /usr/local/lib/libiconv.so.3). Things can | > get interesting when the /usr/local/lib pulls in libc. So now your | > FreeBSD 4 binary could pull in a libiconv.so.3 for FreeBSD 7 and | > libc from FreeBSD 7. Interesting things start to happen! | > | > What I've done for work is to teach the rtld to look for the | > .note.ABI-tag and extract the osversion. I then put that and the | > osversion major number (ie. 6 instead of 603000) into a "try" list. | > Then whenever an object/lib is attemped to be accessed I look in the | > osversion sub-directory then the osversion major and finally the | > standard location. I even extended this to LD_PRELOAD and | > insert this try into the path just before the object/lib if fully | > qualified. Actually, I do it for all fully qualified things. | > Now I can stick most things into /usr/lib/compat/6 and it just | > works. For fully qualified LD_PRELOAD I "mv" things on boot | > into the sub-directory for that OS. Also I had to change LD_PRELOAD | > to accept failure if the thing wasn't found since children in herit | > that and if it a different version that LD_PRELOAD might not be | > applicable. | > | > I don't really see a maintain for rtld since jdp when I worked with | > him to add LD_PRELOAD. I'd like to get feed back on this idea and | > get it into FreeBSD. It would help solve some of these issues for | > other people as well. It definitely, made my life easier at work. | > It would also make things easier on my FreeBSD machines, when I | > upgrade the OS and ports but have legacy compiled things that I | > don't want to recompile again. I could move all of my /usr/local/libs | > into /usr/local/lib/compat/ still run my old stuff. I've had issues before when OO was cross-linked | > with libc from 6 and 7 due to this. Rebuilding OO was not fun and | > I shouldn't have needed to. | > | > There is one glitch in atleast FreeBSD 6.1 didn't put the .note.ABI-tag | > into binaries so for now if not specified I assume FreeBSD 6. That | > is good enough for work but I could add an env. variable to hold | > the default. If not set then it wouldn't do the sub-directory thing | > on binaries that are not known. | > | > Loading of objects will slow done a bit since it adds extra | > searches. It might be good enough to just do the os version major | > and not for the complete osversion. | > | > BTW, I also added a feature to look at LD32_ first then LD_ | > when running 32bit on 64bit. Legacy SW doesn't know about | > LD32_ and just sets LD_. This way that stuff doesn't need | > to be taught about LD32_ when it shouldn't really need to. | | There is a popular feature, unfortunately, not supported by FreeBSD | ld.so, called Dynamic String Tokens, see | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view | | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT | without serious conflicts. | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch That is an interesting feature, however, it almost seems backwards for me if I understand it correctly. I need old binaries to find the library it was built with and not new ones based on the base OS. The plus that I see with their feature is for a library that has been optimized for a specific type of CPU etc. Doug A. From das at FreeBSD.ORG Thu Feb 12 16:18:40 2009 From: das at FreeBSD.ORG (David Schultz) Date: Thu Feb 12 16:18:47 2009 Subject: rtld enhancement to add osversion sub-directory search In-Reply-To: <200902122119.n1CLJOmI092041@ambrisko.com> References: <20090212201101.GI2723@deviant.kiev.zoral.com.ua> <200902122119.n1CLJOmI092041@ambrisko.com> Message-ID: <20090213001935.GA21752@zim.MIT.EDU> On Thu, Feb 12, 2009, Doug Ambrisko wrote: > Kostik Belousov writes: > | There is a popular feature, unfortunately, not supported by FreeBSD > | ld.so, called Dynamic String Tokens, see > | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view > | > | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, > | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT > | without serious conflicts. > | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch > > That is an interesting feature, however, it almost seems backwards for > me if I understand it correctly. I need old binaries to find the library > it was built with and not new ones based on the base OS. The plus that > I see with their feature is for a library that has been optimized for a > specific type of CPU etc. The Solaris rtld features are very useful when you want to export a volume with a bunch of apps over NFS, and the clients are running different releases or different architectures. It can probably also solve your problem if you bother to place different library versions in different directories and set your library path appropriately. As you mention, it's also useful for optimization; people who install binary releases don't need to tolerate libraries that have been compiled to run on an 80486 DX. From brooks at FreeBSD.ORG Thu Feb 12 17:00:54 2009 From: brooks at FreeBSD.ORG (Brooks Davis) Date: Thu Feb 12 17:01:01 2009 Subject: rtld enhancement to add osversion sub-directory search In-Reply-To: <20090213001935.GA21752@zim.MIT.EDU> References: <20090212201101.GI2723@deviant.kiev.zoral.com.ua> <200902122119.n1CLJOmI092041@ambrisko.com> <20090213001935.GA21752@zim.MIT.EDU> Message-ID: <20090213002141.GA47689@lor.one-eyed-alien.net> On Thu, Feb 12, 2009 at 07:19:35PM -0500, David Schultz wrote: > On Thu, Feb 12, 2009, Doug Ambrisko wrote: > > Kostik Belousov writes: > > | There is a popular feature, unfortunately, not supported by FreeBSD > > | ld.so, called Dynamic String Tokens, see > > | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view > > | > > | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, > > | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT > > | without serious conflicts. > > | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch > > > > That is an interesting feature, however, it almost seems backwards for > > me if I understand it correctly. I need old binaries to find the library > > it was built with and not new ones based on the base OS. The plus that > > I see with their feature is for a library that has been optimized for a > > specific type of CPU etc. > > The Solaris rtld features are very useful when you want to > export a volume with a bunch of apps over NFS, and the clients are > running different releases or different architectures. It can > probably also solve your problem if you bother to place different > library versions in different directories and set your library > path appropriately. > > As you mention, it's also useful for optimization; people who > install binary releases don't need to tolerate libraries that have > been compiled to run on an 80486 DX. In principle this would allow numeric libraries like ATLAS to be compiled for all the microarchitectures on a heterogeneous cluster. That could have a major impact on some applications. -- Brooks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090213/f0351059/attachment.pgp From ambrisko at ambrisko.com Fri Feb 13 13:08:59 2009 From: ambrisko at ambrisko.com (Doug Ambrisko) Date: Fri Feb 13 13:09:05 2009 Subject: rtld enhancement to add osversion sub-directory search In-Reply-To: <20090213001935.GA21752@zim.MIT.EDU> Message-ID: <200902132108.n1DL8uG2086318@ambrisko.com> David Schultz writes: | On Thu, Feb 12, 2009, Doug Ambrisko wrote: | > Kostik Belousov writes: | > | There is a popular feature, unfortunately, not supported by FreeBSD | > | ld.so, called Dynamic String Tokens, see | > | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view | > | | > | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, | > | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT | > | without serious conflicts. | > | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch | > | > That is an interesting feature, however, it almost seems backwards for | > me if I understand it correctly. I need old binaries to find the library | > it was built with and not new ones based on the base OS. The plus that | > I see with their feature is for a library that has been optimized for a | > specific type of CPU etc. | | The Solaris rtld features are very useful when you want to | export a volume with a bunch of apps over NFS, and the clients are | running different releases or different architectures. It can | probably also solve your problem if you bother to place different | library versions in different directories and set your library | path appropriately. Unless I missed something it seems to be an inverse feature to what I want since it expands based on the current kernel's uname type results. I need it to expand based on the original OS that it was built on. I'll have to see if my Solaris box at work has this feature or not. I can't change LD_LIBRARY_PATH etc. type things. Doing ldconfig -m of various things doesn't help since that can find the wrong one. My idea was to do something like what happens for 32bit on 64bit and Linux on FreeBSD in that it looks at osversion specific places then the standard. Doug A. From a134qaed at gmail.com Fri Feb 13 13:11:10 2009 From: a134qaed at gmail.com (Dylan Cochran) Date: Fri Feb 13 13:11:16 2009 Subject: rtld enhancement to add osversion sub-directory search In-Reply-To: <20090213002141.GA47689@lor.one-eyed-alien.net> References: <20090212201101.GI2723@deviant.kiev.zoral.com.ua> <200902122119.n1CLJOmI092041@ambrisko.com> <20090213001935.GA21752@zim.MIT.EDU> <20090213002141.GA47689@lor.one-eyed-alien.net> Message-ID: On Thu, Feb 12, 2009 at 7:21 PM, Brooks Davis wrote: > On Thu, Feb 12, 2009 at 07:19:35PM -0500, David Schultz wrote: >> On Thu, Feb 12, 2009, Doug Ambrisko wrote: >> > Kostik Belousov writes: >> > | There is a popular feature, unfortunately, not supported by FreeBSD >> > | ld.so, called Dynamic String Tokens, see >> > | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view >> > | >> > | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, >> > | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT >> > | without serious conflicts. >> > | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch >> > >> > That is an interesting feature, however, it almost seems backwards for >> > me if I understand it correctly. I need old binaries to find the library >> > it was built with and not new ones based on the base OS. The plus that >> > I see with their feature is for a library that has been optimized for a >> > specific type of CPU etc. >> >> The Solaris rtld features are very useful when you want to >> export a volume with a bunch of apps over NFS, and the clients are >> running different releases or different architectures. It can >> probably also solve your problem if you bother to place different >> library versions in different directories and set your library >> path appropriately. >> >> As you mention, it's also useful for optimization; people who >> install binary releases don't need to tolerate libraries that have >> been compiled to run on an 80486 DX. > > In principle this would allow numeric libraries like ATLAS to be > compiled for all the microarchitectures on a heterogeneous cluster. > That could have a major impact on some applications. > > -- Brooks > I'm interested in this work as well. Right now I have all the infrastructure in place to generate images with multiple freebsd kernel versions and architectures, but it relies on a statically linked binary to nullfs mount the right /lib and /libexec into place, depending upon which kernel is booted. So anything that could get rid of a hack like that would be very welcome indeed. :) From brooks at freebsd.org Fri Feb 13 13:32:00 2009 From: brooks at freebsd.org (Brooks Davis) Date: Fri Feb 13 13:32:08 2009 Subject: rtld enhancement to add osversion sub-directory search In-Reply-To: <200902132108.n1DL8uG2086318@ambrisko.com> References: <20090213001935.GA21752@zim.MIT.EDU> <200902132108.n1DL8uG2086318@ambrisko.com> Message-ID: <20090213213042.GA58675@lor.one-eyed-alien.net> On Fri, Feb 13, 2009 at 01:08:56PM -0800, Doug Ambrisko wrote: > David Schultz writes: > | On Thu, Feb 12, 2009, Doug Ambrisko wrote: > | > Kostik Belousov writes: > | > | There is a popular feature, unfortunately, not supported by FreeBSD > | > | ld.so, called Dynamic String Tokens, see > | > | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view > | > | > | > | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, > | > | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT > | > | without serious conflicts. > | > | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch > | > > | > That is an interesting feature, however, it almost seems backwards for > | > me if I understand it correctly. I need old binaries to find the library > | > it was built with and not new ones based on the base OS. The plus that > | > I see with their feature is for a library that has been optimized for a > | > specific type of CPU etc. > | > | The Solaris rtld features are very useful when you want to > | export a volume with a bunch of apps over NFS, and the clients are > | running different releases or different architectures. It can > | probably also solve your problem if you bother to place different > | library versions in different directories and set your library > | path appropriately. > > Unless I missed something it seems to be an inverse feature to > what I want since it expands based on the current kernel's > uname type results. I need it to expand based on the original OS > that it was built on. I'll have to see if my Solaris box at work > has this feature or not. I can't change LD_LIBRARY_PATH etc. > type things. Doing ldconfig -m of various things doesn't help > since that can find the wrong one. My idea was to do something > like what happens for 32bit on 64bit and Linux on FreeBSD in that > it looks at osversion specific places then the standard. While I commented on the Dynamic String Tokens and think it might be useful, that's a distraction for your proposal. I think you proposal sounds useful as well. If it was in 7 by the time I upgrade my cluster from 6, I'd probably use it to ease the transition. -- Brooks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090213/5ec0d642/attachment.pgp From ambrisko at ambrisko.com Fri Feb 13 16:39:27 2009 From: ambrisko at ambrisko.com (Doug Ambrisko) Date: Fri Feb 13 16:39:33 2009 Subject: rtld enhancement to add osversion sub-directory search In-Reply-To: <20090213213042.GA58675@lor.one-eyed-alien.net> Message-ID: <200902140039.n1E0dQ1p098986@ambrisko.com> Brooks Davis writes: -- Start of PGP signed section. | On Fri, Feb 13, 2009 at 01:08:56PM -0800, Doug Ambrisko wrote: | > David Schultz writes: | > | On Thu, Feb 12, 2009, Doug Ambrisko wrote: | > | > Kostik Belousov writes: | > | > | There is a popular feature, unfortunately, not supported by FreeBSD | > | > | ld.so, called Dynamic String Tokens, see | > | > | http://docs.sun.com/app/docs/doc/817-1984/appendixc-4?l=en&a=view | > | > | | > | > | I have almost abandoned patch that adds support for $ORIGIN, $OSREL, | > | > | $OSNAME, and $PLATFORM. Quite amazingly, it merged with today CURRENT | > | > | without serious conflicts. | > | > | http://people.freebsd.org/~kib/misc/rtld_locks.4.patch | > | > | > | > That is an interesting feature, however, it almost seems backwards for | > | > me if I understand it correctly. I need old binaries to find the library | > | > it was built with and not new ones based on the base OS. The plus that | > | > I see with their feature is for a library that has been optimized for a | > | > specific type of CPU etc. | > | | > | The Solaris rtld features are very useful when you want to | > | export a volume with a bunch of apps over NFS, and the clients are | > | running different releases or different architectures. It can | > | probably also solve your problem if you bother to place different | > | library versions in different directories and set your library | > | path appropriately. | > | > Unless I missed something it seems to be an inverse feature to | > what I want since it expands based on the current kernel's | > uname type results. I need it to expand based on the original OS | > that it was built on. I'll have to see if my Solaris box at work | > has this feature or not. I can't change LD_LIBRARY_PATH etc. | > type things. Doing ldconfig -m of various things doesn't help | > since that can find the wrong one. My idea was to do something | > like what happens for 32bit on 64bit and Linux on FreeBSD in that | > it looks at osversion specific places then the standard. | | While I commented on the Dynamic String Tokens and think it might be | useful, that's a distraction for your proposal. I think you proposal | sounds useful as well. If it was in 7 by the time I upgrade my cluster | from 6, I'd probably use it to ease the transition. Since my scheme seems useful, then I'll proceed to clean up my patch some more then send it out for people to play with and comment on. Thanks, Doug A. From bugmaster at FreeBSD.org Mon Feb 16 03:06:49 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Feb 16 03:07:25 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200902161106.n1GB6lHB096049@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From rwatson at FreeBSD.org Mon Feb 16 04:48:24 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Mon Feb 16 04:48:31 2009 Subject: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) In-Reply-To: <20080526110543.J26343@fledge.watson.org> References: <20080526110543.J26343@fledge.watson.org> Message-ID: (Bcc to arch@) On Mon, 26 May 2008, Robert Watson wrote: > Just to keep track of things: > > http://wiki.freebsd.org/NONMPSAFE_DEORBIT Delayed by about six months, the merge and switch to the new USB stack in 8.x means that we're now fairly close to being able to pick up this project again. The goal remains to eliminate IFF_NEEDSGIANT, which is (mostly) the last piece of non-MPSAFE compatibility infrastructure in the network stack in -CURRENT. I removed support for non-MPSAFE network protocols before 7.0, and this is the support for non-MPSAFE network device drivers. As of the current moment in HEAD, the following drivers are flagged wth IFF_NEEDSGIANT: General network device drivers that still require Giant: if_ar if_ray if_sl if_sr Old USB network device drivers: if_axe if_cdce if_cue if_kue if_rue if_rum if_udav if_upgt if_ural if_urtw if_zyd Network device drivers intimately tangled with the old TTY code: if_cx if_ppp lf_sl A network device driver that appears to conditionally use IFF_NEEDSGIANT for the purposes of (sometimes) interacting with the old USB code: if_ndis The following schedule is proposed, assuming nothing goes horribly wrong with the new USB code in the next few weeks, and remaining nits relating to USB network and 802.11 drivers are handled: 16 February 2009 HEADS UP to lists (this e-mail) 01 March 2009 Disable build of all IFF_NEEDSGIANT drivers in 8.x 01 April 2009 Remove all IFF_NEEDSGIANT drivers from 8.x In the next couple of weeks, I'd like to resolve the status of (and eliminate) the if_ndis conditional use of IFF_NEEDSGIANT. There's also a chance that if_sl will get updated by Ed and myself to work with the new locking and TTY world orders -- the lock is easy, but the TTY update takes a bit of work. Perhaps someone will feel moved to do this for if_ppp and possibly if_cx as well. Robert N M Watson Computer Laboratory University of Cambridge _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From rdivacky at freebsd.org Sun Feb 22 11:22:39 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Sun Feb 22 11:22:45 2009 Subject: switching world to gnu99 Message-ID: <20090222190005.GA20129@freebsd.org> hi I'd like to switch world to gnu99 (it's gnu89 now) compilation with the following patch: http://www.vlakno.cz/~rdivacky/c99.patch It sets CSTD to gnu99 and fixes a few abuses of CFLAGS+=-std=something in the tree. The patch was tested with make universe and ports exp build. The main benefits for this are mainly bugs fixing (I've fixed some things by compiling the world in gnu99 mode) and better support for other compilers beside gcc. Please review this (it was already looked at by ruslan) and unless someone strongly objects I'd like to commit this in a week or so. thank you roman -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090222/1abec552/attachment.pgp From bugmaster at FreeBSD.org Mon Feb 23 03:06:49 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Feb 23 03:07:19 2009 Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org Message-ID: <200902231106.n1NB6m3R055437@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From rwatson at FreeBSD.org Tue Feb 24 11:30:02 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Tue Feb 24 12:10:41 2009 Subject: HEADS UP: IFF_NEEDSGIANT consumers to be disabled, removed (was: Re: Wiki page for non-MPSAFE network stack de-orbit scheduling) In-Reply-To: References: <20080526110543.J26343@fledge.watson.org> Message-ID: On Mon, 16 Feb 2009, Robert Watson wrote: > The following schedule is proposed, assuming nothing goes horribly wrong > with the new USB code in the next few weeks, and remaining nits relating to > USB network and 802.11 drivers are handled: > > 16 February 2009 HEADS UP to lists (this e-mail) > 01 March 2009 Disable build of all IFF_NEEDSGIANT drivers in 8.x > 01 April 2009 Remove all IFF_NEEDSGIANT drivers from 8.x > > In the next couple of weeks, I'd like to resolve the status of (and > eliminate) the if_ndis conditional use of IFF_NEEDSGIANT. There's also a > chance that if_sl will get updated by Ed and myself to work with the new > locking and TTY world orders -- the lock is easy, but the TTY update takes a > bit of work. Perhaps someone will feel moved to do this for if_ppp and > possibly if_cx as well. Just a reminder that 1 March is gradually approaching. It looks like the new USB stack is settling nicely, so I currently have no plans to defer the above schedule. Robert N M Watson Computer Laboratory University of Cambridge From news at shapevine.com Wed Feb 25 02:10:22 2009 From: news at shapevine.com (Shapvine News) Date: Wed Feb 25 02:10:29 2009 Subject: Shapevine: News from the Vine - OPT-OUT Confirmation Message-ID: <20090225101009.302.1220427499.swift@www.ecastnews.com> Your unsubscription to our mailing list, Shapevine: News from the Vine, has been initiated. To verify your unsubscription, please visit: http://www.ecastnews.com/listServer/box.php?ai=&mi=29&p=0&e=YXJjaEBmcmVlYnNkLm9yZw==&funcml=cunsub&nl=15 Any questions may be sent to our mailing list admin at news@shapevine.com From ed at 80386.nl Fri Feb 27 05:11:57 2009 From: ed at 80386.nl (Ed Schouten) Date: Fri Feb 27 05:12:03 2009 Subject: Making LLVM happy: memmove() in the kernel Message-ID: <20090227131155.GE19161@hoeg.nl> Hi all, The FreeBSD+LLVM folks* noticed Clang generates calls to memmove() by itself. I have yet to confirm this, but I assume this is done when performing copies of structs greater than a certain size. In our kernel, we don't have a memmove() function, but we do have a bcopy(). Because memmove() must be a function in this case (not a simple macro), Roman and I agreed that adding a memmove() to libkern would be the best thing to do for now, simply by calling bcopy(). ARM already has a memmove() in support.S, so we don't need it there. So my question is: what is your folks opinion on this patch? http://80386.nl/pub/memmove.diff It would be lovely if we could integrate this patch (or a similar one), because this will allow us to build kernels with Clang out of the box. -- Ed Schouten WWW: http://80386.nl/ * http://wiki.freebsd.org/BuildingFreeBSDWithClang -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090227/3f22dc5f/attachment.pgp From rdivacky at freebsd.org Fri Feb 27 05:33:48 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Fri Feb 27 05:33:56 2009 Subject: Making LLVM happy: memmove() in the kernel In-Reply-To: <20090227131155.GE19161@hoeg.nl> References: <20090227131155.GE19161@hoeg.nl> Message-ID: <20090227131221.GA60215@freebsd.org> On Fri, Feb 27, 2009 at 02:11:55PM +0100, Ed Schouten wrote: > Hi all, > > The FreeBSD+LLVM folks* noticed Clang generates calls to memmove() by > itself. I have yet to confirm this, but I assume this is done when > performing copies of structs greater than a certain size. In our kernel, > we don't have a memmove() function, but we do have a bcopy(). also.. quoting from (http://gcc.gnu.org/onlinedocs/gcc-4.2.1/gcc/Standards.html): Most of the compiler support routines used by GCC are present in libgcc, but there are a few exceptions. GCC requires the freestanding environment provide memcpy, memmove, memset and memcmp. we were just lucky to not run into this -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090227/0bc0b051/attachment.pgp From avg at icyb.net.ua Fri Feb 27 08:05:42 2009 From: avg at icyb.net.ua (Andriy Gapon) Date: Fri Feb 27 08:05:49 2009 Subject: Making LLVM happy: memmove() in the kernel In-Reply-To: <20090227131221.GA60215@freebsd.org> References: <20090227131155.GE19161@hoeg.nl> <20090227131221.GA60215@freebsd.org> Message-ID: <49A80F4D.8000406@icyb.net.ua> on 27/02/2009 15:12 Roman Divacky said the following: > On Fri, Feb 27, 2009 at 02:11:55PM +0100, Ed Schouten wrote: >> Hi all, >> >> The FreeBSD+LLVM folks* noticed Clang generates calls to memmove() by >> itself. I have yet to confirm this, but I assume this is done when >> performing copies of structs greater than a certain size. In our kernel, >> we don't have a memmove() function, but we do have a bcopy(). > > also.. quoting from > (http://gcc.gnu.org/onlinedocs/gcc-4.2.1/gcc/Standards.html): > > Most of the compiler support routines used by GCC are present in libgcc, > but there are a few exceptions. GCC requires the freestanding environment > provide memcpy, memmove, memset and memcmp. > > we were just lucky to not run into this Some people actually were not that lucky and had to use similar workarounds. -- Andriy Gapon From scf at FreeBSD.org Fri Feb 27 08:40:52 2009 From: scf at FreeBSD.org (Sean C. Farley) Date: Fri Feb 27 08:40:58 2009 Subject: Making LLVM happy: memmove() in the kernel In-Reply-To: <20090227131155.GE19161@hoeg.nl> References: <20090227131155.GE19161@hoeg.nl> Message-ID: On Fri, 27 Feb 2009, Ed Schouten wrote: > Hi all, > > The FreeBSD+LLVM folks* noticed Clang generates calls to memmove() by > itself. I have yet to confirm this, but I assume this is done when > performing copies of structs greater than a certain size. In our > kernel, we don't have a memmove() function, but we do have a bcopy(). > > Because memmove() must be a function in this case (not a simple > macro), Roman and I agreed that adding a memmove() to libkern would be > the best thing to do for now, simply by calling bcopy(). ARM already > has a memmove() in support.S, so we don't need it there. > > So my question is: what is your folks opinion on this patch? > > http://80386.nl/pub/memmove.diff > > It would be lovely if we could integrate this patch (or a similar > one), because this will allow us to build kernels with Clang out of > the box. Does bcopy() in the kernel allow for overlapping strings? Sean -- scf@FreeBSD.org From scf at FreeBSD.org Fri Feb 27 08:45:25 2009 From: scf at FreeBSD.org (Sean C. Farley) Date: Fri Feb 27 08:45:32 2009 Subject: Making LLVM happy: memmove() in the kernel In-Reply-To: References: <20090227131155.GE19161@hoeg.nl> Message-ID: On Fri, 27 Feb 2009, Sean C. Farley wrote: *snip* > Does bcopy() in the kernel allow for overlapping strings? *sigh* Never mind the sleepy programmer. When will the coffee kick in? :) Sean -- scf@FreeBSD.org From kabaev at gmail.com Fri Feb 27 10:52:13 2009 From: kabaev at gmail.com (Alexander Kabaev) Date: Fri Feb 27 10:52:19 2009 Subject: Making LLVM happy: memmove() in the kernel In-Reply-To: <49A80F4D.8000406@icyb.net.ua> References: <20090227131155.GE19161@hoeg.nl> <20090227131221.GA60215@freebsd.org> <49A80F4D.8000406@icyb.net.ua> Message-ID: <20090227132242.4ef6a633@kan.dnsalias.net> On Fri, 27 Feb 2009 18:05:33 +0200 Andriy Gapon wrote: > on 27/02/2009 15:12 Roman Divacky said the following: > > On Fri, Feb 27, 2009 at 02:11:55PM +0100, Ed Schouten wrote: > >> Hi all, > >> > >> The FreeBSD+LLVM folks* noticed Clang generates calls to memmove() > >> by itself. I have yet to confirm this, but I assume this is done > >> when performing copies of structs greater than a certain size. In > >> our kernel, we don't have a memmove() function, but we do have a > >> bcopy(). > > > > also.. quoting from > > (http://gcc.gnu.org/onlinedocs/gcc-4.2.1/gcc/Standards.html): > > > > Most of the compiler support routines used by GCC are present in > > libgcc, but there are a few exceptions. GCC requires the > > freestanding environment provide memcpy, memmove, memset and memcmp. > > > > we were just lucky to not run into this > > Some people actually were not that lucky and had to use similar > workarounds. > I think we should use this opportunity and make sure we have external symbols for all of the above mem* functions, not just memmove. Please :) -- Alexander Kabaev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20090227/cbcbfcc1/signature.pgp From rdivacky at freebsd.org Sat Feb 28 01:58:31 2009 From: rdivacky at freebsd.org (Roman Divacky) Date: Sat Feb 28 01:58:38 2009 Subject: Making LLVM happy: memmove() in the kernel In-Reply-To: <20090228175724.D14305@delplex.bde.org> References: <20090227131155.GE19161@hoeg.nl> <20090227131221.GA60215@freebsd.org> <49A80F4D.8000406@icyb.net.ua> <20090227132242.4ef6a633@kan.dnsalias.net> <20090228175724.D14305@delplex.bde.org> Message-ID: <20090228095502.GA62459@freebsd.org> On Sat, Feb 28, 2009 at 06:43:54PM +1100, Bruce Evans wrote: > On Fri, 27 Feb 2009, Alexander Kabaev wrote: > > >On Fri, 27 Feb 2009 18:05:33 +0200 > >Andriy Gapon wrote: > > > >>on 27/02/2009 15:12 Roman Divacky said the following: > >>>On Fri, Feb 27, 2009 at 02:11:55PM +0100, Ed Schouten wrote: > >>>>Hi all, > >>>> > >>>>The FreeBSD+LLVM folks* noticed Clang generates calls to memmove() > >>>>by itself. I have yet to confirm this, but I assume this is done > >>>>when performing copies of structs greater than a certain size. In > > Why would Clang be that broken? Structs cannot overlap, so they can structs can overlap: struct foo { double x[100]; }; void T(struct foo *F, struct foo* G) { *F = *G; } when F equals to G From brde at optusnet.com.au Sat Feb 28 04:32:27 2009 From: brde at optusnet.com.au (Bruce Evans) Date: Sat Feb 28 04:32:39 2009 Subject: Making LLVM happy: memmove() in the kernel In-Reply-To: <20090227132242.4ef6a633@kan.dnsalias.net> References: <20090227131155.GE19161@hoeg.nl> <20090227131221.GA60215@freebsd.org> <49A80F4D.8000406@icyb.net.ua> <20090227132242.4ef6a633@kan.dnsalias.net> Message-ID: <20090228175724.D14305@delplex.bde.org> On Fri, 27 Feb 2009, Alexander Kabaev wrote: > On Fri, 27 Feb 2009 18:05:33 +0200 > Andriy Gapon wrote: > >> on 27/02/2009 15:12 Roman Divacky said the following: >>> On Fri, Feb 27, 2009 at 02:11:55PM +0100, Ed Schouten wrote: >>>> Hi all, >>>> >>>> The FreeBSD+LLVM folks* noticed Clang generates calls to memmove() >>>> by itself. I have yet to confirm this, but I assume this is done >>>> when performing copies of structs greater than a certain size. In Why would Clang be that broken? Structs cannot overlap, so they can be copied by memcpy() which should be faster. (However, in the FreeBSD kernel, use of memcpy() is only approved for fixed-sized copies small enough to be inlined, so memcpy() is only a placeholder for the non-inlined case and for abuse, and extensive optimizations are only applied to bcopy() and bcopy() might be faster; however2, the optimizations don't work for any current hardware so memcpy() should be faster than bcopy() because it is simpler. The inlining also fails in all cases since _all_ gcc builtins were turned off many years as a side effect of using -ffreestanding (see below for more details). The loss was so insignficant that no one noticed it. Even now, with the placeholders for memcpy() and memset() abused a lot, fixing their inlining makes no significant difference.) >>>> our kernel, we don't have a memmove() function, but we do have a >>>> bcopy(). >>> >>> also.. quoting from >>> (http://gcc.gnu.org/onlinedocs/gcc-4.2.1/gcc/Standards.html): >>> >>> Most of the compiler support routines used by GCC are present in >>> libgcc, but there are a few exceptions. GCC requires the >>> freestanding environment provide memcpy, memmove, memset and memcmp. >>> >>> we were just lucky to not run into this Since this is a bug in gcc, we were unlucky to run into it. These functions are in the application namespace in freestanding environments. Compiler support functions must be named something like __memmove(). FreeBSD started using -ffreestanding mainly to avoid related bugs. The gcc builtin for printf() sometimes translates printf() to use puts() and perhaps any function in stdio. FreeBSD has printf() but not puts() in the kernel so linkage started failing when gcc implemented builtin printf (similarly for a couple of other standard library functions). The problem was fixed by compiling all FreeBSD kernels with -ffreestanding, which was implemented in gcc at about the same time that builtin printf was implemented. -ffreestanding must kill all standard library functions. But them it necessarily kills all builtins for such functions and thus prevents automatic inlining of everything in . >> Some people actually were not that lucky and had to use similar >> workarounds. > > I think we should use this opportunity and make sure we have external > symbols for all of the above mem* functions, not just memmove. Please :) How do you make them fully external so that they are only used by broken external software? Bruce