svn - but smaller?
Ian Smith
smithi at nimnet.asn.au
Sat Mar 16 15:14:42 UTC 2013
On Wed, 13 Mar 2013 21:11:28 -0500, John Mehr wrote:
> On Wed, 13 Mar 2013 16:29:45 +1100 (EST)
> Ian Smith <smithi at nimnet.asn.au> wrote:
[..]
> > I have a small test system on which I'd installed (two instances of) 9.1 so
> > a couple of days ago I fetched ports with portsnap, installed svnup, and
> > ran it using the (just what I needed) example command in svnup(1).
Just to reiterate: this is vanilla releng/9.1, no ports but docs and
svnup installed, no X, 512MB RAM, nothing happening but occasional ARP.
For completeness, it's running powerd and acpi_ibm but not cam.ctl.
> > I get about 700KB/s here, and svnup took about 15 minutes to update 9.1
> > sources to 9-stable. This is fine. Last night I ran it again, but it took
> > 12:42 to make no changes. This seemed puzzling, as you'd said only a few
> > minutes for subsequent updates, but the reason appears to be that in both
> > cases, I ran it in script(1), and the default verbosity of 1 includes a
> > listing of every directory and file examined, followed by <CR> then <erase
> > to eol> codes. Even in less -r (raw) mode it still has to display and skip
> > through all the (now invisible) lines; bit messy.
> >
> > Even the second do-nothing run made a 2MB script file, the original with
> > all 9.1 to -stable updates being 3.4MB. So I'd love the option to only
> > list the changes (- and +) and simply ignore unchanged dirs/files without
> > any display for use in script(1). Apart from that, I'm happy.
>
> Which mirror are you using? I ran several tests tonight repeatedly fetching
> 9/stable from svn0.us-west (so they would all be do-nothing runs) both inside
> and outside of a script(1) capture and on both an old SSD and on a ZFS
> mirrored array (to see if the target media made any difference) and they all
> completed in 2 minutes, 43 seconds +/- 2 seconds on my 350 KB/s connection.
svn0.us-west.freebsd.org as per example, but it's the closest to here
anyway; pings avg 217.5ms stddev 0.4ms, us-east avg 269.3ms sd 9.5ms.
fastest_cvsup used to find cvs mirrors in Australia at around 55-70ms,
so for another while I thought RTT might be the issue - but it's not.
> I'll definitely put in a verbosity level that does exactly what you suggest.
> Sorry about that.
Not at all, and thanks! But neither, I now find, is logging the extra
time issue; as mentioned elsewhere, running at -v0 makes next to no
difference, 12:48 west vs 12:54 east on 4 do-nothing runs, 700KB/s link,
not running in script(1) - though I think that matters not, or little.
So I began watching top more closely, also running iostat -w10, noticing
that svnup was pegging CPU at 100% with ~75% _system_? time for nearly 9
minutes of the run, doing little or no disk I/O and a max of 130KB/s in,
22.6KB/s out, no stress there, growing to ~20MB resident, then doing a
flurry of disk I/O at about 3-5MB/s for the last 3m40s odd. IOW, svnup
here is mostly heavily CPU-bound, which I don't understand, but then I
know nothing about what level of computation the protocol demands.
I've done some more controlled tests since, running both iostat -w10 and
netstat -w10 -dn before, during and until a bit after the run, attached,
hoping these might help pinpoint any bottlenecks? This was running -v1
default verbosity, hence the (not at all heavy) tty output numbers.
Just sometimes, running slower hardware can have advantages! On your
times, yours likely has more than 500% of my single-core P3-M's 1133MHz
performance, which would tend to mask this issue pretty well for you :)
John Baldwin has best described when c{,v}sup deletes (only in-tree)
files, covered in section 'delete' in csup(1). I think this should
probably be default behaviour, even if svn proper would do otherwise.
cheers, Ian
-------------- next part --------------
input (Total) output
packets errs idrops bytes packets errs bytes colls drops
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
312 0 0 504213 212 0 50511 0 0
587 0 0 758448 384 0 108299 0 0
858 0 0 1307014 601 0 139574 0 0
699 0 0 777451 521 0 111510 0 0
612 0 0 821160 429 0 99541 0 0
459 0 0 653828 331 0 106364 0 0
453 0 0 510009 336 0 94317 0 0
371 0 0 440457 275 0 114857 0 0
341 0 0 409886 273 0 80247 0 0
345 0 0 467728 254 0 115426 0 0
324 0 0 390130 262 0 79802 0 0
337 0 0 433303 253 0 82200 0 0
355 0 0 406168 282 0 113992 0 0
322 0 0 416256 239 0 114022 0 0
320 0 0 381180 258 0 79670 0 0
input (Total) output
packets errs idrops bytes packets errs bytes colls drops
329 0 0 423802 248 0 114353 0 0
327 0 0 421522 259 0 80446 0 0
318 0 0 353895 261 0 112430 0 0
311 0 0 384996 232 0 113088 0 0
164 0 0 156731 147 0 60615 0 0
544 0 0 798317 398 0 226807 0 0
611 0 0 673035 481 0 156500 0 0
337 0 0 410430 251 0 113920 0 0
324 0 0 451341 256 0 81325 0 0
340 0 0 391423 256 0 113631 0 0
330 0 0 430792 261 0 80809 0 0
320 0 0 367279 264 0 112839 0 0
304 0 0 382159 225 0 113268 0 0
305 0 0 346967 247 0 78712 0 0
324 0 0 373735 244 0 112784 0 0
283 0 0 369634 235 0 112957 0 0
300 0 0 361895 247 0 79332 0 0
297 0 0 331210 243 0 111691 0 0
291 0 0 382309 245 0 113146 0 0
349 0 0 450901 250 0 81294 0 0
351 0 0 427975 262 0 114521 0 0
input (Total) output
packets errs idrops bytes packets errs bytes colls drops
353 0 0 432793 285 0 81325 0 0
336 0 0 434476 245 0 81003 0 0
339 0 0 391010 254 0 113304 0 0
329 0 0 416502 267 0 80605 0 0
340 0 0 459859 250 0 115134 0 0
361 0 0 431423 281 0 81099 0 0
305 0 0 364965 256 0 113085 0 0
266 0 0 310331 223 0 111339 0 0
297 0 0 362677 217 0 112503 0 0
347 0 0 438560 273 0 80977 0 0
361 0 0 438921 269 0 114764 0 0
340 0 0 427522 268 0 80788 0 0
349 0 0 448910 261 0 115090 0 0
339 0 0 413140 271 0 80508 0 0
333 0 0 449658 245 0 81374 0 0
358 0 0 421497 287 0 114393 0 0
331 0 0 431352 237 0 80610 0 0
180 0 0 149490 137 0 41963 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
input (Total) output
packets errs idrops bytes packets errs bytes colls drops
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
2 0 0 192 2 0 174 0 0
1 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0
input (Total) output
packets errs idrops bytes packets errs bytes colls drops
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
-------------- next part --------------
tty ada0 cd0 pass0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 250 10.49 3 0.03 0.00 0 0.00 0.00 0 0.00 1 0 1 0 98
0 31 32.00 0 0.01 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 15 2.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 15 11.33 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 15 2.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
2 36 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 4 0 14 0 82
0 2685 5.86 86 0.49 0.00 0 0.00 0.00 0 0.00 19 0 62 0 19
0 4960 6.63 116 0.75 0.00 0 0.00 0.00 0 0.00 14 0 49 0 37
0 4320 4.10 26 0.10 0.00 0 0.00 0.00 0 0.00 18 0 69 0 12
0 2976 4.00 1 0.00 0.00 0 0.00 0.00 0 0.00 18 0 75 0 7
0 5175 30.11 26 0.77 0.00 0 0.00 0.00 0 0.00 20 0 78 0 2
1 2996 3.91 2 0.01 0.00 0 0.00 0.00 0 0.00 19 0 79 0 2
2 3882 16.16 4 0.06 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 6353 32.00 9 0.29 0.00 0 0.00 0.00 0 0.00 29 0 71 0 0
0 4485 30.00 1 0.02 0.00 0 0.00 0.00 0 0.00 26 0 74 0 0
0 6080 5.20 0 0.00 0.00 0 0.00 0.00 0 0.00 29 0 71 0 0
0 4200 1.50 0 0.00 0.00 0 0.00 0.00 0 0.00 24 0 76 0 0
0 6300 12.67 30 0.37 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 4188 27.00 0 0.01 0.00 0 0.00 0.00 0 0.00 24 0 76 0 0
0 6356 23.78 1 0.02 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
tty ada0 cd0 pass0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 4376 3.33 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 6530 30.67 0 0.01 0.00 0 0.00 0.00 0 0.00 24 0 75 0 0
0 5447 10.00 0 0.00 0.00 0 0.00 0.00 0 0.00 26 0 74 0 0
0 5619 3.33 0 0.00 0.00 0 0.00 0.00 0 0.00 22 0 78 0 0
0 6705 2.67 0 0.00 0.00 0 0.00 0.00 0 0.00 27 0 73 0 0
0 4403 21.33 0 0.01 0.00 0 0.00 0.00 0 0.00 24 0 76 0 0
0 15092 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 33 0 67 0 0
0 4526 3.20 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 6334 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 4541 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 24 0 76 0 0
0 6051 2.00 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 4444 12.00 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 6572 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 6604 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 27 0 73 0 0
0 4422 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 22 0 77 0 0
0 6805 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 24 0 76 0 0
0 6888 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 23 0 77 0 0
0 6942 17.85 2 0.03 0.00 0 0.00 0.00 0 0.00 26 0 74 0 0
0 4774 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 22 0 78 0 0
0 6163 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 26 0 74 0 0
tty ada0 cd0 pass0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 3966 20.00 0 0.01 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 5727 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 30 0 70 0 0
0 4021 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 24 0 76 0 0
0 4040 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 6254 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 26 0 74 0 0
0 4001 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 5997 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 4226 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 23 0 77 0 0
0 6735 4.00 0 0.00 0.00 0 0.00 0.00 0 0.00 24 0 76 0 0
0 6666 2.00 0 0.00 0.00 0 0.00 0.00 0 0.00 21 0 78 0 0
0 6258 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 3999 6.00 0 0.00 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 6163 2.00 0 0.00 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 4107 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 5850 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 4265 6.00 0 0.00 0.00 0 0.00 0.00 0 0.00 25 0 75 0 0
0 4170 16.00 0 0.00 0.00 0 0.00 0.00 0 0.00 26 0 74 0 0
0 6002 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 28 0 72 0 0
0 6877 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 23 0 73 0 3
0 213 13.22 329 4.24 0.00 0 0.00 0.00 0 0.00 10 0 7 1 82
tty ada0 cd0 pass0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 239 30.57 160 4.77 0.00 0 0.00 0.00 0 0.00 10 0 6 0 84
0 281 13.96 292 3.98 0.00 0 0.00 0.00 0 0.00 8 0 6 0 85
1 41 9.02 441 3.89 0.00 0 0.00 0.00 0 0.00 8 0 11 1 80
2 32 13.62 351 4.67 0.00 0 0.00 0.00 0 0.00 15 0 9 0 76
1 17 12.69 462 5.72 0.00 0 0.00 0.00 0 0.00 10 0 12 1 77
1 254 21.81 250 5.33 0.00 0 0.00 0.00 0 0.00 12 0 8 0 79
0 119 13.50 353 4.66 0.00 0 0.00 0.00 0 0.00 10 0 9 1 80
0 125 16.60 241 3.91 0.00 0 0.00 0.00 0 0.00 9 0 7 1 84
0 131 13.94 270 3.68 0.00 0 0.00 0.00 0 0.00 8 0 6 1 85
0 111 17.80 159 2.76 0.00 0 0.00 0.00 0 0.00 6 0 4 0 89
0 104 10.98 233 2.50 0.00 0 0.00 0.00 0 0.00 5 0 5 0 90
0 111 9.38 283 2.59 0.00 0 0.00 0.00 0 0.00 7 0 6 0 86
0 118 10.16 201 1.99 0.00 0 0.00 0.00 0 0.00 7 0 4 0 88
0 145 7.15 416 2.90 0.00 0 0.00 0.00 0 0.00 7 0 9 1 83
0 119 7.59 452 3.35 0.00 0 0.00 0.00 0 0.00 7 0 11 1 81
0 106 7.50 285 2.09 0.00 0 0.00 0.00 0 0.00 5 0 6 1 89
0 126 20.29 189 3.75 0.00 0 0.00 0.00 0 0.00 11 0 5 0 84
0 121 27.75 132 3.58 0.00 0 0.00 0.00 0 0.00 13 0 5 0 82
0 117 16.99 205 3.40 0.00 0 0.00 0.00 0 0.00 12 0 5 1 83
0 114 16.71 172 2.80 0.00 0 0.00 0.00 0 0.00 9 0 4 0 87
tty ada0 cd0 pass0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 226 21.91 185 3.95 0.00 0 0.00 0.00 0 0.00 9 0 6 0 84
0 92 9.75 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 76 24.51 6 0.14 0.00 0 0.00 0.00 0 0.00 0 0 1 0 99
1 68 8.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
6 84 31.33 1 0.02 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
1 82 15.43 1 0.01 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 70 13.12 0 0.01 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
More information about the freebsd-stable
mailing list