Bursty data transfer with Dummynet

[Ahmed Hamza]

Thank you Luigi and Julian. I believe I figured it out. It was totally my bad.

I configured Dummynet with the default queue size (50 slots) and a
very high bandwidth on the FreeBSD LiveCD as Luigi suggested, and
everything was working. Then I realized what the problem was. I'm
testing an adaptive streaming application in which the bitrate (and
consequently size) of the requested data changes based on bandwidth
estimation. For some reason there is a problem with my bandwidth
estimation logic and the returned value is an overestimation.
Therefore it takes time to fill up the application buffers and the
buffered content are played back quickly. There are no bursts during
the download process itself.

I haven't tried the Ubuntu guest again yet. But I will check it to see
the effect of having a kernel configured with 250Hz.

-Ahmed

On Wed, Nov 13, 2013 at 8:06 PM, Luigi Rizzo <rizzo at iet.unipi.it> wrote:
>
>
>
> On Wed, Nov 13, 2013 at 6:06 AM, Ahmed Hamza <ahmed.hmz at gmail.com> wrote:
>>
>> On Tue, Nov 12, 2013 at 8:50 PM, Julian Elischer <julian at freebsd.org>
>> wrote:
>> > On 11/12/13, 6:35 PM, Ahmed Hamza wrote:
>> >>
>> >> Hi All,
>> >>
>> >> I'm trying to use Dummynet to test the behaviour of my video streaming
>> >> application in various network conditions. Dummynet was compiled and
>> >> installed on an Ubuntu 12.04 box with a 2.6 Linux kernel. I'm
>> >> experiencing a strange behaviour when I reduce the bandwidth of the
>> >> link/path.
>> >>
>> >> For some reason, instead of having a slow download speed. It seems as
>> >> if the download is happening in bursts! A portion of the data is
>> >> downloaded at a high speed, then the data transfer stops for a period
>> >> of time and then resumes again (and so on). Does anyone have an idea
>> >> what could be the cause? Or is this even an expected behaviour? If so,
>> >> why?
>> >
>> >
>> > I can't really speak for dummynet on Linux but the granularity of the
>> > queues
>> > is dependent on the timer granularity of the kernel you have and to some
>> > extent will rely on the correct integration of dummynet into the timer
>> > facility of the kernel you are running it in. On freeBSD with a 1kHz
>> > 'tick'
>> > I'd' expect to see packets being release from the queue each mSec or so.
>> > if you are getting second sized chunks then it probably is a bug. Either
>> > dummynet is not compatible with that kind of kernel, something else has
>> > gone
>> > wrong.  It COULD also be that you are catching the wrong packets.. I've
>> > seen
>> > similar behaviour when I was accidentally queuing all the acks instead
>> > of
>> > all the data going in the other direction, but I presume you have
>> > already
>> > checked to see what you are queuing.
>> >
>>
>> Thanks Julian and Matthew for your replies. To clarify my settings,
>> below are the outputs from 'ipfw show' and 'ipfw pipe show'. I should
>> also mention that it seems the by default the Ubuntu kernel is
>> configured for 250Hz, not 1000Hz.
>>
>> root at vm1:~/# ipfw pipe 1 config bw 500Kbit/s queue 500Kbyte
>>
>> root at nsl-vm1:~/# ipfw show
>> 00100  202247  105063701 pipe 1 ip from 192.168.56.4 to any
>> 65535 2245577 2648958386 allow ip from any to any
>>
>> root at nsl-vm1:~/# ipfw pipe show
>> 00001: 500.000 Kbit/s    0 ms  500 KB 1 queues (1 buckets) droptail
>>     mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
>> BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes
>> Pkt/Byte Drp
>>   0 tcp     192.168.56.1/33547    192.168.56.4/80    238083 134515909
>> 0    0 623
>
>
> to debug the problem i'd first try removing/reducing the queue size,
> because you are queueing  8 seconds of data and that may interact
> badly with whatever flow control protocol you have implemented
> (even if you are using tcp underneath, you might have some logic
> in your app that decides when to send, etc.)
>
> Also, try to start with a very high bandwidth and gradually
> reduce it and see if there is a point where behaviour suddenly changes.
>
> As julian pointed out, dummynet works on a tick, but at the rates
> you are using the granularity should not matter much.
>
> And, to answer another poster, modern hardware is pretty good at
> running VMs with millisecond or less ticks.
>
> cheers
> luigi
>