Socket rate limiting (was: Re: making nmdm(4) emulate actual speed.)

[Luigi Rizzo]

two comments: the reasons why apps implement throttling on their
own are
1) portability - they cannot assume the required services
   are available on the platforms they are going to run on;
2) scheduling and resource management - apps have their own
   requirement on how to schedule things and it is often
   unlikely that they can express them in term of altq/dummynet
   "classes".

This said, there is no problem in having thousands of dummynet
pipes -- the algorithms used in dummynet have O(log N) cost
where N is the number of active pipes.

cheers
luigi

On Sat, Jan 15, 2005 at 01:36:07PM +0300, Yar Tikhiy wrote:
...
> A lot of network daemons implement rate limits by their own.  For
> instance, Apache httpd has mod_throttle, Squid has delay pools, and
> even lukemftpd has some sort of rate limiting code in it.  With the
> demand for rate limiting being so high, it could be natural for an
> OS to provide such service to applications through, e.g., a per-socket
> option.
> 
> However, I've got an impression from the first glance at the issue
> that it would be next to impossible to implement such service in a
> fashion independent of lower layers of network abstraction.  I
> suppose that we have such elaborate and IP-centric subsystems as
> DUMMYNET and ALTQ partly because of the complexity of the task.
> Therefore a possible approach is to implement the rate-limit socket
> option using one of those existing subsystems.  Such solution would
> scale poorly though--imagine a server with thousands of rate-limited
> sockets open, each of them requiring a separate DUMMYNET pipe.  Did
> anybody have other thoughts or see publications regarding the
> problem?
> 
> -- 
> Yar