Question About TCP Reassembly Inside VImages

[Jason Fines]

Hi Jorge,

Sorry about the delay in my response.  I have also been able to set maxqlen,
but have been unsuccessful in setting maxsegments.  I just recently tried
upgrading to 7_RELENG with the vimage package
http://imunes.tel.fer.hr/virtnet/vimage_7-20081015.tgz however I am still
experiencing the same problems.  I even went so far as to set the variable
'V_tcp_reass_maxseg' explicitly to 4096 on lines 111, and 122 of tcp_reass.c
in the kernel source code, however, the maxsegments sysctl variable is still
zero in my vimages!!!

Have you tried anything else?  I am also prohibited from using this amazing
technology because of this issue.

Does anyone else have any information about this?  Marko, would upgrading
FreeBSD 8.0 make any difference?

Any help would be greatly appreciated.

Thanks,
Jason

On Thu, Nov 27, 2008 at 11:57 AM, Jorge Sanchez <jorgesanchez75 at live.com>wrote:

>
>
>
>
> Hola Jason,
>
> I also observed a similar phenomenon on my system's vimages. I have several
> thousands dropped packets due to "insufficient memory" (the counter you
> mention in netstat -m for me is also incremented in the
> net.inet.tcp.reass.overflows read-only sysctl variable) and I routinely have
> TCP connections dropped within vimages because of it. I think that the TCP
> packet reassembly queue length is essentially zero once options VIMAGE is
> enabled... which would explain my problems when trying to contact hosts that
> are on flaky links or are situated very far away hop-wise.
>
> I think there is something very wrong with the TCP reassembly when options
> VIMAGE is enabled. Did you try increasing the net.inet.reass.maxqlen or
> net.inet.reass.maxsegments sysctls? I was able to increase maxqlen but
> maxsegments must be set in loader.conf and this value is not inherited into
> any vimages I create after booting! :-(
>
> If you come up with a fix, I would appreciate it too since this prevents me
> from performing realistic TCP testing within the virtualization framework.
>
> Adios!
> Jorge
>
>
> From: jason.fines at gmail.com
> To:
>          freebsd-virtualization at freebsd.org
> Subject: Re: Question About TCP Reassembly Inside VImages
> Date: Sat, 22 Nov 2008 08:52:16 -0500
> Thanks Julian,
>
> I suspect you are correct as nmbclusters is a system wide sysctl variable
> set at boot time and although V_tcp_reass_maxseg is set per vimage it is
> the
> result of a constant operation done on nmbclusters (nmbclusters / 16).
>
> What I've described is what I suspect is the root of my problem.  The
> manifestation of this problem is that TCP packets passing through my
> vimage(s) are not reassembled when they are out of order and I get an
> exceptionally high value reported by netstat -m stating that packets were
> dropped due to "insufficient memory".  Posts I've found on the net point to
> the reassembly queue length, which in the vimages is zero for some reason.
>
> Perhaps this additional information will help clarify my exact problem.
>
> Thanks,
> Jason
>
> On Sat, Nov 22, 2008 at 5:12 AM, Julian Elischer <julian at elischer.org
> >wrote:
>
> > Jason Fines wrote:
> >
> >> Hello all,
> >>
> >> I've got a question about setting the sysctl variable
> >> net.inet.tcp.reass.maxsegments to a non-zero value inside my vimages.
>  I'm
> >> currently running the FreeBSD 7 with the VIMAGE package available at
> >> http://imunes.tel.fer.hr/virtnet/vimage_7-20081015.tgz.
> >>
> >> My problem is with TCP reassembly support inside of the vimages, namely
> >> with
> >> the tcp.reass.maxsegments sysctl variable.  I've tracked down where in
> the
> >> code the variable is set to line 122 in tcp_reass_init() of
> >> netinet/tcp_reass.c: "V_tcp_reass_maxseg = nmbclusters / 16;".  The line
> >> clearly reads that maxsegments should be set to "nmbclusters /16", in
> the
> >> main OS (not in any vimage) the value is correctly set to 1/16 of what
> my
> >> nmbclusters sysctl variable is set to.  However, inside all my vimages
> >> nmbclusters is set correctly, while reass.maxsegments is incorrectly set
> >> to
> >> zero!!!
> >>
> >
> > V_tcp_reass_maxseg is a macro that hides the fact that
> > tcp_reass_maxseg is a PER Vimage variable.
> >
> > Part of the patch
> > is to make some sysctls be per-vimage. I do not know exactly
> > about that one.. I suspect it is actually a read-only
> > whole-system value, and not per vimage.
> >
> >
> >
> >
> >
> >> Is it possible that nmbclusters when read on line 122 of
> >> netinet/tcp_reass.c
> >> is zero?  Has anyone else experienced this problem?  Is TCP reassembly
> not
> >> supported/tested inside vimages?
> >>
> >> Any help in this area would be greatly appreciated.
> >>
> >> Thanks,
> >> Jason
> >>
> >> P.S. This technology is phenomenal, and thanks to everyone who is
> involved
> >> developing it.
> >> _______________________________________________
> >> freebsd-virtualization at freebsd.org mailing list
> >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> >> To unsubscribe, send any mail to "
> >> freebsd-virtualization-unsubscribe at freebsd.org"
> >>
>
>
> _________________________________________________________________
>
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe at freebsd.org"
>