Flow-tools and Flow-tools-ng - Flow-Capture - supposed to be memory leak - patch

[Zvezdelin Vladov]

Hello Stanislav,

According to the author of the patch:
"Paul,

Upon further investigation, it appears as if there is an alignment issue in
'struct msgip' inside of 'struct ftnet'.  I've looked at some other code, and
there appear to be a handful of useful CMSG_* macros to help deal with this.
RFC 2292 details them.

The attached patch works cleanly on my system and fixes the issue at hand.
Hopefully it doesn't break other platforms.  It's there for anybody who is
interested.

Thanks,
Mark"

I'll try to describe what I saw from the debugging of the flow-tools
code (not the NG branch, but the
regular one) and from the patch provided. That is how I understand it.

So, in the first - original source of the flow-tools code, there were
some structures
that (re)declared the IP header, and dealt with it directly. While
this worked for the IPv4, 32bit
code, the same actions applied to 64code base, doesn't work. And the
whole code, that
finds different exporters, doesn't work. The whole scene is changed
then - while you should
have at most say - 10-60 different exporters, now because the IP
header isn't decoded correctly
you have almost millions of different exporters, and for each of these
exporters the code
takes memory.

And it starts to look like a regular memory leak -- but in reality it
is totally different problem - IP header
decoding problem.

And In the patch, there is a new method deployed to describe and
decode the IP header, and the whole
method is implemented by the regular work with specially purposed
macro-es -- taken from RFC 2299
-- Advanced Sockets API for IPv6  (1998).

So, in my opinion, it is perfectly suitable patch for the FreeBSD to
include it. After all it does what it really
has to do - to patch a problem with the IP header decoding. And patch
it the portable way.

Hope this helps.

BR,

Z.Vladov

Of course, you may think differently.

On Sun, Sep 27, 2009 at 3:11 PM, Stanislav Sedov <stas at freebsd.org> wrote:
> On Sat, 15 Aug 2009 10:04:46 +0300
> Zvezdelin Vladov <zvladov at gmail.com> mentioned:
>
>> Dear Sir/Madam,
>>
>> Please, publish in the official ports build patch system, the patch
>> bellow for the problem
>> that manifest itself only on the amd64 platform, and it is one and the
>> same for both the
>> flow-tools and flow-tools-ng - i.e. flow-capture eats all of the RAM
>> and SWAP of the
>> machine, until killed by the kernel.
>>
>> The problem itself is in the built-in mechanism in the flow-tools,(
>> instead of using the
>> macros in the FreeBSD,) to read msg structures and control-information
>> from recvmsg
>> calls.
>>
>> I've managed to patch myself the source, but my patch is ugly one, and
>> consist of
>> just commenting out the usage of the hack-in tools to read msg structures of the
>> original source (but worked).
>>
>> The patch I am pasting here is from the author, and concerns the same problem.
>> It looks much better than mine, and If I've new about it, I would be
>> spending 4 days
>> debugging....;-(
>>
>
> Hello, Zvezdelin!
>
> Can you elaborate a bit more on the patch contents?  I'm not entirely sure
> what it does and furthermore flow-tools source code does not contain any
> references to the CMSG_XXX macros, so it is effectively a no-op for
> net/flow-tools.  It might work for flow-tools-ng though.
>
> It'd be helpful to have a small description of what these changes does,
> so it will be possible to elaborate the same fix for net/flow-tools.
>
> --
> Stanislav Sedov
> ST4096-RIPE
>