PPPoE server (high traffic in WDM network)

[Дмитрий Замураев]

Hello, Michelle

>The main problem with the bandwidth is, that even the VOD/IPTV and  VoIP
>traffic goes throug the PPPoE server which is very bad.
I don't think so, yes it is bad and not very bad. It depends on BRAS
hardware.

>I do not want to count the traffic to a  specific  /25  which  hold  the
>storage servers,If you want this, you may use the MPD-specific RADIUS
attributes described in:
http://mpd.sourceforge.net/doc5/mpd30.html#30
see last paragraph.
> mean, the VOD/IPTV and VoIP  traffic  must  bypass  the
>PPPoE server.
If VOD/IPTV is uses multicast - yes else you must set the pipe for this kind 
of traffic
>  and this reduce the traffic enormous...
hmm, What better way for you:
change all active network devices (e.g. switches) to use the multicast 
groups
or buy additional servers for PPPoE purpose

>My idea is/was, to put the PPPoE server diretly byside the FTTH DSLAM's,
>which mean, each 96port DSLAM has an upstrem of 1 GE and even if  I  put
>10 of them in a 42RU, it would normaly not fill the  10 GE  ports  of  a
>professionel Server.  And of corse,  I  can  put  always  two  or  three
>together parallel.
Yes, it is good idea.

>The problem is only, that I can not install 10 (or 20 redunant) 1U Sun
>Fire X4100M2, even if I can get up to 60% rebat of the listprice.
>I have not the place to put 20 additiona servers into, nor  I  like  the
>power consumation ~70 Watt with the smalles CPU and only 4 GByte of RAM.
I have't Sun servers and i can't compare Sun with i386 usage with this task
I use i386 servers. See below.

>> instead of bigger one
>> gives better results performance-wise. You also have to test if SMP
>> helps and how much. A beast with 16 cores is more powerful from a
>> regular computer with 2 cores, but does it help in your setup?
> If I go with 1 U Sun Fire X4100M2 the Opteron has 4 Cores
> and 4  threads per core (AFAIK there is a 8 threads version too)
the SMP with many NIC helps, and so much

>> Can you recomment it for an ISP setup?
>> It's FreeBSD running from a read-only mounted medium.
>> No more, no less. Yes, it's fine for an ISP setup.
>If I have 4 GByte of memory, I could run entirely from RAMDISK...
>Memory is cheaper then the harddrives
I use usb flash drive in read-write mode as full replace of hard drive



The *BIG* problem is to handle rx packets which goes in to pppoe server.

The goal of which is using hardware interrupt moderation of NIC and good
motherboard.
Maybe second cache of CPU to be more and more is better (but i have't tests)


I use that configuration:
The hardware:
S3000AHV motherboard with Core2Quad 6600 processor (4 cores).
motherboard has 82571 chip NIC, the two additional cars: EXPI9402
(PCI-E x4 two 10/100/1000BaseTX ports) and EXPI9400 (PCI-E x1
one ethernet port) is installed to the system.
The goal of which is to load all cores of CPU
or use yandex em(4) driver who uses different rx kernel threads on every
CPU/core (if you have one NIC)
for the driver see: people. yandex . ru/~wawa
The software:
1. FreeBSD 7.0, mpd5.3 + ipfw + dummynet (shaping on this machine - bad
    idea! for performance.  but it is first my setup and works very stable)
2. FreeBSD 7.2, mpd5.3 + ng_car + ng_bpf (rate-limiting
    on this machine - the perfomance of traffic processing is very good,
    because ng_car & ng_bpf is in kernel space, but have trouble
    described in PR kern/137881, i think on FBSD 7.0 it may be works fine,
    but no test are made, i'm busy)
3. Use PPPoE concentrator and traffic shaper/rate_limiting on different
    machines - it is best but is very older practice.
I think the best choise today is #2



Dmitriy Zamuraev, system administrator
The Netline NSP company
Russia, Tyumen.