svn - but smaller?
Jeremy Chadwick
jdc at koitsu.org
Sun Feb 24 06:31:13 UTC 2013
On Sun, Feb 24, 2013 at 05:44:10AM +0100, Michael Ross wrote:
> On Sun, 24 Feb 2013 05:16:38 +0100, Jeremy Chadwick <jdc at koitsu.org> wrote:
>
> >On Sun, Feb 24, 2013 at 04:56:23AM +0100, Michael Ross wrote:
> >>On Sun, 24 Feb 2013 04:15:09 +0100, Jeremy Chadwick
> >><jdc at koitsu.org> wrote:
> >>
> >>>On Sun, Feb 24, 2013 at 03:45:57AM +0100, Michael Ross wrote:
> >>>>On Sun, 24 Feb 2013 01:36:36 +0100, John Mehr <jcm at visi.com> wrote:
> >>>>
> >>>>> Hello all,
> >>>>> I've believe I've made just about all of the progress
> >>>>optimizing svnup
> >>>>> as I can and I've just submitted it as a new port. With my
> >>>>~ 350kb/s
> >>>>> DSL connection, it now takes just under 30 minutes to
> >>>>download a fresh
> >>>>> base/releng/8.3 tree using svnup (Subversion's svn takes
> >>>>approximately
> >>>>> 12 minutes). Incremental updates, such as tracking one of
> >>>>the stable
> >>>>> branches takes only 2-3 minutes.
> >>>>> For anyone that wants to preview the port before it gets
> >>>>added to the
> >>>>> ports tree (assuming I got the send-pr correct), the tarball is
> >>>>>located
> >>>>> at:
> >>>>> http://jcm.dsl.visi.com/freebsd/svnup/svnup-0.5.tar.xz
> >>>>> Please let me know if you find any issues.
> >>>>
> >>>>
> >>>>Maybe it's me, but:
> >>>>
> >>>>No Makefile.
> >>>>So I try manually:
> >>>>
> >>>>gurder> cc svnup.c
> >>>>/tmp//cconKEOv.o: In function `compare_md5':
> >>>>svnup.c:(.text+0x175e): undefined reference to `MD5Init'
> >>>>svnup.c:(.text+0x1774): undefined reference to `MD5Update'
> >>>>svnup.c:(.text+0x1785): undefined reference to `MD5End'
> >>>>/tmp//cconKEOv.o: In function `get_files':
> >>>>svnup.c:(.text+0x1f20): undefined reference to `MD5Init'
> >>>>svnup.c:(.text+0x1f4e): undefined reference to `MD5Update'
> >>>>svnup.c:(.text+0x1f5f): undefined reference to `MD5End'
> >>>>
> >>>>
> >>>>9.0-STABLE FreeBSD 9.0-STABLE #17: Fri May 4 02:53:49 CEST 2012
> >>>
> >>>Those are all defined in libmd (see MD5Init(3) man page). Thus:
> >>>
> >>>cc -o svnup svnup.c -lmd
> >>>
> >>
> >>Thanks.
> >>
> >>gurder> ./svnup -h svn0.us-west.FreeBSD.org -b ports/head -l test
> >>
> >>Dumps core: http://gurder.ross.cx/misc/svnup.core
> >
> >Should be easy enough to debug:
> >
> >$ cc -g3 -ggdb -o svnup svnup.c -lmd
> >$ gdb svnup
> >(gdb) run -h svn0.us-west.FreeBSD.org -b ports/head -l test
> >
> >Then once it cores, provide the output from "bt" and/or "bt full".
> >
> >Might also be useful to compile with -Wall to see if there are any
> >compile-time warnings.
> >
>
> gurder> cc -Wall -g3 -ggdb -o svnup svnup.c -lmd
> svnup.c: In function 'main':
> svnup.c:1002: warning: zero-length printf format string
> svnup.c:1020: warning: zero-length printf format string
> svnup.c:1027: warning: zero-length printf format string
> svnup.c:1065: warning: zero-length printf format string
>
>
> (gdb) run -h svn0.us-west.FreeBSD.org -b ports/head -l test
> Starting program: /usr/ports/sysutils/svnup-0.5/svnup -h
> svn0.us-west.FreeBSD.org -b ports/head -l test
> ####### Fetching revision: 312860
> ? test/archivers
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000000800b22950 in strchr () from /lib/libc.so.7
>
> Backtrace: http://gurder.ross.cx/misc/backtrace.txt
I downloaded it and looked at the source.
svnup.c:1002: warning: zero-length printf format string
svnup.c:1020: warning: zero-length printf format string
svnup.c:1027: warning: zero-length printf format string
svnup.c:1065: warning: zero-length printf format string
1002 sprintf(command, "");
1020 sprintf(command, "");
1027 sprintf(command, "");
1065 sprintf(command, "");
I'm not sure what the intention is here. If it's to terminate the
buffer, then all of these should be:
command[0] = '\0';
Or if the entire command[] buffer needs to be zeroed, use memset(3).
Doing this does not fix the segfault.
The segfault experienced is caused by the following problem:
Program received signal SIGSEGV, Segmentation fault.
0x00000000a0b2eb10 in strchr () from /lib/libc.so.7
(gdb) bt
#0 0x00000000a0b2eb10 in strchr () from /lib/libc.so.7
#1 0x00000000004021e8 in build_source_directory_tree (connection=0x7fffffff4820, command=0x7ffffffc3650 "", file=0xa1135000,
file_count=0x7fffffff48f8, max_file=0x7fffffff48fc, path_target=0xa1009058 "test", revision=0) at svnup.c:412
#2 0x0000000000000000 in ?? ()
(gdb) f 1
#1 0x00000000004021e8 in build_source_directory_tree (connection=0x7fffffff4820, command=0x7ffffffc3650 "", file=0xa1135000,
file_count=0x7fffffff48f8, max_file=0x7fffffff48fc, path_target=0xa1009058 "test", revision=0) at svnup.c:412
412 end = strchr(directory, '\n');
(gdb) p directory
$1 = 0x0
It's a wee bit hard to do strchr() on something that points to NULL.
At line 412 I inserted this, which is almost certainly not
sufficient/correct:
if (directory == NULL) {
continue;
}
This allows the program to get a bit further than before (same goes if I
use break instead of continue), but not before segfaulting again. And
segfaults after this point contain a completely smashed stack:
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x0000000000000000 in ?? ()
John will need to look into these. They all reek of string parsing
anomalies and so on. I would need to sit down and read/understand the
code in full to know how to fix this one.
Also, John, please consider using malloc(3) instead of heap-allocated
buffers like file_buffer[6][] (196608 bytes) and command[] (32769
bytes). I'm referring to this:
47 #define COMMAND_BUFFER 32768
386 char new_path_target[BUFFER_UNIT], file_buffer[6][COMMAND_BUFFER], *path_source;
836 char *start, *value, *addr, *branch, *path_target, temp_file[BUFFER_UNIT], command[COMMAND_BUFFER + 1];
I should also point out that watching this thing in top(1) shows RES/RSS
increasing until the segfault (for me dies out around 4.5MBytes). I
don't know if that's by design or not, but I don't see why this thing
would need so much memory given what it's doing. You'd know better than
I though.
You may want to consider running this under valgrind. It's remarkable
what you can find with that during debugging.
--
| Jeremy Chadwick jdc at koitsu.org |
| UNIX Systems Administrator http://jdc.koitsu.org/ |
| Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
More information about the freebsd-stable
mailing list