network bitrate of a poll of processes

Mathieu Prevot mathieu.prevot at gmail.com
Mon May 26 22:45:21 UTC 2008


2008/5/26 Marc Spitzer <mspitzer at gmail.com>:
> On Mon, May 26, 2008 at 12:14 PM, Mathieu Prevot
> <mathieu.prevot at gmail.com> wrote:
>> 2008/5/26 Dan Nelson <dnelson at allantgroup.com>:
>>> In the last episode (May 26), Mathieu Prevot said:
>>>> Hi,
>>>>
>>>> I would like to know the bitrate of a pool of child processes that use
>>>> a network connection, how can I have something like netstat -w1
>>>> provide but at the process level ?
>>>
>>> If you can segregate them to their own UID, you can use an ipfw "count
>>> ip from any to any uid 6666" rule to count the packets.  Another option
>>> would be to jail them to a dedicated IP address and count traffic on
>>> that IP.
>>
>> Both are nice... thank you. And don't you have something more portable
>> ie. that could run on "any" posix os - like a sh script that use
>> standard tools -  and that doesn't need root user intervention ?
>>
>> Mathieu
>
> I do not think so, firewall for example is controlled by root.  And
> users do not generally have detailed access to other users
> information.  You can look into the BASM framework to track what your
> apps are doing and if you know the behavior to expect you can use
> netflow/argus/tcpdump to track that and post process the logs in each
> case into something useful.
>
> What are you trying to do, details do help?
>
> marc

I have a script that generates many child processes that download
videos (~10MB) in this case, or do heavy data processing/analysing. I
can tune the number of workers as I want, but It's a shared cluster so
I would like to

1) measure 2) optimize/control (by number of workers ? at the script
level ! so I can update thing very dynamically)
a) CPU load b) IO on drive(s) c) network load

and I prefer solutions that don't need root access/intervention. I can
measure indirectly network load using `date +%s` and `du -b |sed
's/\t.*//' ` regularly... but I may have to deal with several 10GB
later - and actually the number of files is more critical than their
size.
Maybe writing a (very) quick and dirty python app that listen/attach
to a group of processes (for the network case) ?

Mathieu


More information about the freebsd-hackers mailing list