From stephen at missouri.edu Tue Aug 4 18:17:56 2009 From: stephen at missouri.edu (Stephen Montgomery-Smith) Date: Tue Aug 4 18:18:02 2009 Subject: Pushing files onto ftp-master Message-ID: <4A7878B5.5020907@missouri.edu> I am the person in charge of maintaining CTM (CVSUP via email). Currently I create the CTM files on my home machine. A few times a day, ftp-master uses rsync to extract the CTM files from my machine, and from there it goes out to the internet. I am now being told that I may need to get behind a firewall in the next few months. Is there a mechanism where I could rsync, as the client, onto ftp-master? That is, instead of ftp-master initiating the rsync connection, that I initiate the rsync connection? Thanks, Stephen From stephen at missouri.edu Wed Aug 12 16:57:17 2009 From: stephen at missouri.edu (Stephen Montgomery-Smith) Date: Wed Aug 12 16:57:23 2009 Subject: Cannot cvsup the whole tree Message-ID: <4A82F46B.6030200@missouri.edu> When I try to CVSUP the whole tree, I keep getting errors like this: ........ Edit ports/lang/icon/Makefile,v Add delta 1.34 2009.08.12.14.55.16 amdmi3 Create src/contrib/ipfilter/ip_irc_pxy.c,v -> Attic Detailer failed: Network read failure: Connection timed out Will retry at 11:41:08 From john.marshall at riverwillow.com.au Fri Aug 14 08:42:57 2009 From: john.marshall at riverwillow.com.au (John Marshall) Date: Fri Aug 14 08:43:04 2009 Subject: Australian CVSup Mirror Statistics Message-ID: <20090814082745.GQ6527@rwpc12.mby.riverwillow.net.au> AUSTRALIAN FREEBSD MIRROR SERVERS --------------------------------- Australian FTP Mirror Servers: 3 Australian WWW Mirror Servers: 2 Australian CVSup Mirror Servers: 1 AUSTRALIAN FREEBSD CVSUP SERVER ------------------------------- It is now just over 12 months since we set up a third Australian CVSup mirror server for the FreeBSD Project. A few weeks later it was the only one left standing. All of the cvsup domain names in au.FreeBSD.org point to one server. I have been accumulating statistics from the cvsupd log. A few months ago I managed to get around to learning about RRDtool and graphing those statistics. The graphs on this page are updated daily with totals from the previous day's cvsupd log file and display statistics for the previous nine months. MORE AUSTRALIAN CVSUP MIRROR SERVERS WELCOME! --------------------------------------------- I know some of you like graphs and will be interested to have a look but I'm hoping also to evoke some interest in the Australian FreeBSD community for operators of additional public CVSup mirror servers. If you have been thinking about hosting a CVSup mirror but have been lacking the bandwidth information required to make a decision, the statistics page referenced above is just what you need! Please bear in mind that these statistics represent the entire Australian CVSup mirror load: additional mirror servers can expect to pick up a proportion of the load shown here. I look forward to hearing from volunteers! I'm happy to answer questions and help get additional mirror servers on-line. -- John Marshall -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hubs/attachments/20090814/fde4d94c/attachment.pgp From john.marshall at riverwillow.com.au Sat Aug 15 01:34:49 2009 From: john.marshall at riverwillow.com.au (John Marshall) Date: Sat Aug 15 01:34:56 2009 Subject: CVSup src file Edits almost alway triggering Checksum mismatch and Fixup Message-ID: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> A dialup user of our CVSup mirror alerted me to this. These days, almost every file updated in the src tree triggers a Checksum mismatch, full file download and Fixup. I'm not seeing this for doc, ports, etc.; just src. For example (from the logs of our synch's with cvsup-master)... --------------------------------------- Edit ports/math/jags/Makefile,v Edit ports/math/jags/distinfo,v Edit src/release/Makefile,v src/release/Makefile,v: Checksum mismatch -- will transfer entire file Create src/release/svnbranch.awk,v Edit src/sys/amd64/amd64/local_apic.c,v src/sys/amd64/amd64/local_apic.c,v: Checksum mismatch -- will transfer entire file Edit src/sys/amd64/include/apicvar.h,v src/sys/amd64/include/apicvar.h,v: Checksum mismatch -- will transfer entire file Edit src/sys/amd64/include/pmc_mdep.h,v src/sys/amd64/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file Edit src/sys/dev/hwpmc/hwpmc_core.c,v Edit src/sys/dev/hwpmc/hwpmc_piv.c,v src/sys/dev/hwpmc/hwpmc_piv.c,v: Checksum mismatch -- will transfer entire file Edit src/sys/dev/hwpmc/hwpmc_ppro.c,v src/sys/dev/hwpmc/hwpmc_ppro.c,v: Checksum mismatch -- will transfer entire file Edit src/sys/dev/hwpmc/hwpmc_x86.c,v src/sys/dev/hwpmc/hwpmc_x86.c,v: Checksum mismatch -- will transfer entire file Edit src/sys/i386/i386/local_apic.c,v src/sys/i386/i386/local_apic.c,v: Checksum mismatch -- will transfer entire file Edit src/sys/i386/include/apicvar.h,v src/sys/i386/include/apicvar.h,v: Checksum mismatch -- will transfer entire file Edit src/sys/i386/include/pmc_mdep.h,v src/sys/i386/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file Edit src/sys/pci/agp_i810.c,v -> Attic src/sys/pci/agp_i810.c,v: Checksum mismatch -- will transfer entire file --------------------------------------- Followed by... --------------------------------------- Applying fixups for collection cvs-all/cvs Fixup src/release/Makefile,v Fixup src/sys/amd64/amd64/local_apic.c,v Fixup src/sys/amd64/include/apicvar.h,v Fixup src/sys/amd64/include/pmc_mdep.h,v Fixup src/sys/dev/hwpmc/hwpmc_piv.c,v Fixup src/sys/dev/hwpmc/hwpmc_ppro.c,v Fixup src/sys/dev/hwpmc/hwpmc_x86.c,v Fixup src/sys/i386/i386/local_apic.c,v Fixup src/sys/i386/include/apicvar.h,v Fixup src/sys/i386/include/pmc_mdep.h,v Fixup src/sys/pci/agp_i810.c,v -> Attic --------------------------------------- I am not seeing Fixups for anything other than the src tree. I've checked back through the past month's logs and the story is the same. Is this a consequence of SVN export to CVS? Can it be fixed? Am I just doing something wrong? -- John Marshall -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-hubs/attachments/20090815/8b67c9d2/attachment.pgp From Andre.Albsmeier at siemens.com Mon Aug 17 12:59:49 2009 From: Andre.Albsmeier at siemens.com (Andre Albsmeier) Date: Mon Aug 17 12:59:56 2009 Subject: CVSup src file Edits almost alway triggering Checksum mismatch and Fixup In-Reply-To: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> References: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> Message-ID: <20090817123425.GA15105@curry.mchp.siemens.de> On Sat, 15-Aug-2009 at 11:34:41 +1000, John Marshall wrote: > A dialup user of our CVSup mirror alerted me to this. > > These days, almost every file updated in the src tree triggers a > Checksum mismatch, full file download and Fixup. I'm not seeing this > for doc, ports, etc.; just src. Me too ;-) The problem started (for me) a few month ago. I see the "Checksum mismatch" messages as well. Sometimes two or three, sometimes 100 or more, sometimes none. I used CVSup for years. I tried csup, nothing changed. I switched servers I CVSup from, even tried cvsup.freebsd.org. I tried compressing and non-compressing mode. Like John, I only see the problem on src, not elsewhere. Any ideas? Thanks, -Andre > > For example (from the logs of our synch's with cvsup-master)... > > --------------------------------------- > Edit ports/math/jags/Makefile,v > Edit ports/math/jags/distinfo,v > Edit src/release/Makefile,v > src/release/Makefile,v: Checksum mismatch -- will transfer entire file > Create src/release/svnbranch.awk,v > Edit src/sys/amd64/amd64/local_apic.c,v > src/sys/amd64/amd64/local_apic.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/amd64/include/apicvar.h,v > src/sys/amd64/include/apicvar.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/amd64/include/pmc_mdep.h,v > src/sys/amd64/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/dev/hwpmc/hwpmc_core.c,v > Edit src/sys/dev/hwpmc/hwpmc_piv.c,v > src/sys/dev/hwpmc/hwpmc_piv.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/dev/hwpmc/hwpmc_ppro.c,v > src/sys/dev/hwpmc/hwpmc_ppro.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/dev/hwpmc/hwpmc_x86.c,v > src/sys/dev/hwpmc/hwpmc_x86.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/i386/i386/local_apic.c,v > src/sys/i386/i386/local_apic.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/i386/include/apicvar.h,v > src/sys/i386/include/apicvar.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/i386/include/pmc_mdep.h,v > src/sys/i386/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/pci/agp_i810.c,v -> Attic > src/sys/pci/agp_i810.c,v: Checksum mismatch -- will transfer entire file > --------------------------------------- > > Followed by... > > --------------------------------------- > Applying fixups for collection cvs-all/cvs > Fixup src/release/Makefile,v > Fixup src/sys/amd64/amd64/local_apic.c,v > Fixup src/sys/amd64/include/apicvar.h,v > Fixup src/sys/amd64/include/pmc_mdep.h,v > Fixup src/sys/dev/hwpmc/hwpmc_piv.c,v > Fixup src/sys/dev/hwpmc/hwpmc_ppro.c,v > Fixup src/sys/dev/hwpmc/hwpmc_x86.c,v > Fixup src/sys/i386/i386/local_apic.c,v > Fixup src/sys/i386/include/apicvar.h,v > Fixup src/sys/i386/include/pmc_mdep.h,v > Fixup src/sys/pci/agp_i810.c,v -> Attic > --------------------------------------- > > I am not seeing Fixups for anything other than the src tree. I've > checked back through the past month's logs and the story is the same. > Is this a consequence of SVN export to CVS? Can it be fixed? Am I just > doing something wrong? > > -- > John Marshall -- Never argue with an idiot. They drag you down to their level, then beat you with their experience. From simon at FreeBSD.org Tue Aug 18 18:47:36 2009 From: simon at FreeBSD.org (Simon L. Nielsen) Date: Tue Aug 18 18:47:47 2009 Subject: CVSup src file Edits almost alway triggering Checksum mismatch and Fixup In-Reply-To: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> References: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> Message-ID: <20090818184734.GA1634@arthur.nitro.dk> On 2009.08.15 11:34:41 +1000, John Marshall wrote: > These days, almost every file updated in the src tree triggers a > Checksum mismatch, full file download and Fixup. I'm not seeing this > for doc, ports, etc.; just src. > [...] > Edit src/release/Makefile,v > src/release/Makefile,v: Checksum mismatch -- will transfer entire file Hey, This is unfortunatly a known problem. The problem seems to be that the actually diff sections in the ,v files are at time re-oredered and that breaks CVSup's assumptions about the ,v files. The ,v are still valid in both before and after reorder, the order is just changed. Since cvsup's normal acelerated CVS mirror mode works by knowning how thw ,v files are layed out, cvsup then can just see that the ,v file does not look like it expects, and it does the safe thing and transfer/sync the entire file. I actually suspect if CVSup didn't do this but just transfered the delta's to the ,v files, the resulting ,v file on the client would be different, but working and valid - but this is a guess. kib@ (AFAIR) first noticed that this happened for security updates which went to multiple branches at once in the spring, but it was still only for a few files. After the upgrade of CVS to 1.12 in base (and therefor on repoman - the master CVS server) it seems to be that the reason all files are now have this problem - so CVS 1.12 probably made it worse. I suspect the reason it happens for src, but not ports or doc, is the branches. When there are no branches in a ,v file the ordering is simply newest -> oldest, but when there are branches I don't think the order is clear. So, in short I don't see any easy solutions to this, and especially given the fact that in the long run CVS/CVSup will probably be replaced by something else for distributing FreeBSD source, I don't think it will be that easy to find someone to "fix" either CVSup or CVS. PS. Just to be clear CVS/CVSup is not going away any time soon (though there are no timeline planed yet), but 3-5 years down the line I really doubt we will be using CVSup. -- Simon L. Nielsen Hats: FreeBSD.org cluster, cvs, and svn admin teams From gad at FreeBSD.org Tue Aug 18 20:36:21 2009 From: gad at FreeBSD.org (Garance A Drosehn) Date: Tue Aug 18 20:36:28 2009 Subject: CVSup src file Edits almost alway triggering Checksum mismatch and Fixup In-Reply-To: <20090818184734.GA1634@arthur.nitro.dk> References: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> <20090818184734.GA1634@arthur.nitro.dk> Message-ID: At 8:47 PM +0200 8/18/09, Simon L. Nielsen wrote: >On 2009.08.15 11:34:41 +1000, John Marshall wrote: > >> These days, almost every file updated in the src tree triggers a >> Checksum mismatch, full file download and Fixup. I'm not seeing this >> for doc, ports, etc.; just src. >> >[...] >> Edit src/release/Makefile,v >> src/release/Makefile,v: Checksum mismatch -- will transfer entire file > >Hey, > >This is unfortunatly a known problem. The problem seems to be that >the actually diff sections in the ,v files are at time re-oredered >and that breaks CVSup's assumptions about the ,v files. > >The ,v are still valid in both before and after reorder, the order >is just changed. [...] >After the upgrade of CVS to 1.12 in base (and therefor on repoman - >the master CVS server) it seems to be that the reason all files are >now have this problem - so CVS 1.12 probably made it worse. > >I suspect the reason it happens for src, but not ports or doc, is the >branches. When there are no branches in a ,v file the ordering is >simply newest -> oldest, but when there are branches I don't think >the order is clear. Hmm. I've noticed this too, and assumed that it has something to do with the SVN->CVS conversion process (which is why it would not effect ports or doc). When I've skimmed over some of my cvsup logs, I noticed it doesn't happen for all files. I've been meaning to check what is noteable about the changes where the whole file has to be transfered compared to those where it doesn't. -- Garance Alistair Drosehn = drosehn@rpi.edu Senior Systems Programmer or gad@FreeBSD.org Rensselaer Polytechnic Institute; Troy, NY; USA From dougb at FreeBSD.org Mon Aug 24 22:58:10 2009 From: dougb at FreeBSD.org (Doug Barton) Date: Mon Aug 24 22:58:16 2009 Subject: Request: Please hook up IPv6 to cvsup mirrors Message-ID: <4A9314B7.90405@FreeBSD.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 Howdy, I'm a big proponent of using IPv6 to do as much as humanly possible for a lot of reasons, which I don't think I need to list unless you really want me to. :) My understanding is that cVsup does not have the ability to use IPv6 transport natively, however the csup binary in the base does, and more and more users are migrating to it as time goes on. I've also noticed that the cvsup mirrors that I use most often all have IPv6 addresses associated with them. I see this on startup: Connecting to cvsup10.freebsd.org Cannot connect to 2001:4f8:fff6::30: Connection refused Connected to 69.147.83.48 Now here comes the request. My understanding is that there is a fairly simple netcat trick to link the cvsup port on the IPv6 address to that of the IPv4 address. If that actually is possible, I'd like to request that y'all actually do that. :) I will of course be glad to assist with this in any necessary manner. Doug - -- This .signature sanitized for your protection -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEAREDAAYFAkqTFLcACgkQyIakK9Wy8PuJOQCdE3hD+HJ78iyHaPnswjMQuTA3 0KEAn0sKGtM8VDlqmXyaEGit7vUNNUyb =VAUc -----END PGP SIGNATURE----- From plosher at isc.org Mon Aug 24 23:14:00 2009 From: plosher at isc.org (Peter Losher) Date: Mon Aug 24 23:14:07 2009 Subject: Request: Please hook up IPv6 to cvsup mirrors In-Reply-To: <4A9314B7.90405@FreeBSD.org> References: <4A9314B7.90405@FreeBSD.org> Message-ID: <4A931EB7.6000808@isc.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Doug Barton wrote: > Now here comes the request. My understanding is that there is a fairly > simple netcat trick to link the cvsup port on the IPv6 address to that > of the IPv4 address. If that actually is possible, I'd like to request > that y'all actually do that. :) FWIW, cvsup4{.us}.freebsd.org already does this... - -Peter - -- [ plosher@isc.org | ISC | OpenPGP 0xE8048D08 | "The bits *must* flow" ] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkqTHrcACgkQPtVx9OgEjQi7jwCgtKlBlJ2dQhAMiFHxIYnDa6ld HhMAn2gPRi+zSZcb1e9/+5ho2tsrJCu6 =vVD9 -----END PGP SIGNATURE----- From olli at lurza.secnetix.de Tue Aug 25 07:58:52 2009 From: olli at lurza.secnetix.de (Oliver Fromme) Date: Tue Aug 25 07:58:58 2009 Subject: Request: Please hook up IPv6 to cvsup mirrors In-Reply-To: <4A9314B7.90405@FreeBSD.org> Message-ID: <200908250758.n7P7wYQn097862@lurza.secnetix.de> Doug Barton wrote: > Now here comes the request. My understanding is that there is a fairly > simple netcat trick to link the cvsup port on the IPv6 address to that > of the IPv4 address. If that actually is possible, I'd like to request > that y'all actually do that. :) You can also use the 6tunnel tool (ports/net/6tunnel), which is fairly easy to use: # 6tunnel -6 5999 127.0.0.1 5999 If everything else fails, you can even use FreeBSD's base ssh(1) to create v6-to-v4 tunnels, provided that the encryption overhead is no problem for you (or use a patched openssh that supports "-c none"): # ssh -gNR '[your_v6_address]/5999/127.0.0.1/5999' 127.0.0.1 (You also need to enable the GatewayPorts option in your sshd_config(5) file, then restart sshd(8).) Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Gesch?ftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M?n- chen, HRB 125758, Gesch?ftsf?hrer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "In My Egoistical Opinion, most people's C programs should be indented six feet downward and covered with dirt." -- Blair P. Houghton From mohacsi at niif.hu Tue Aug 25 08:59:04 2009 From: mohacsi at niif.hu (Mohacsi Janos) Date: Tue Aug 25 08:59:11 2009 Subject: Request: Please hook up IPv6 to cvsup mirrors In-Reply-To: <4A9314B7.90405@FreeBSD.org> References: <4A9314B7.90405@FreeBSD.org> Message-ID: On Mon, 24 Aug 2009, Doug Barton wrote: > I've also noticed that the cvsup mirrors that I use most often all > have IPv6 addresses associated with them. I see this on startup: > > Connecting to cvsup10.freebsd.org > Cannot connect to 2001:4f8:fff6::30: Connection refused > Connected to 69.147.83.48 > cvsup.hu.freebsd.org handling both IPv6 and IPv4 - IPv6 with netcat trick. Best Regards, Janos Mohacsi From dougb at FreeBSD.org Tue Aug 25 22:58:05 2009 From: dougb at FreeBSD.org (Doug Barton) Date: Tue Aug 25 22:58:11 2009 Subject: Request: Please hook up IPv6 to cvsup mirrors In-Reply-To: References: <4A9314B7.90405@FreeBSD.org> Message-ID: <4A946C75.40302@FreeBSD.org> Mohacsi Janos wrote: > > > > On Mon, 24 Aug 2009, Doug Barton wrote: > >> I've also noticed that the cvsup mirrors that I use most often all >> have IPv6 addresses associated with them. I see this on startup: >> >> Connecting to cvsup10.freebsd.org >> Cannot connect to 2001:4f8:fff6::30: Connection refused >> Connected to 69.147.83.48 >> > > cvsup.hu.freebsd.org handling both IPv6 and IPv4 - IPv6 with netcat trick. Thanks to you and Oliver for the responses. Re cvsup.hu I would prefer to try and be a good netizen and keep my packets on this side of the Atlantic. :) I use cvsup4.us (ISC) and cvsup10.us (Yahoo!) the most, with the latter as the default since cvsup4 is so often at capacity. Doug -- This .signature sanitized for your protection