freebsd tuning

[Calomel Org]

Kes,

First, understand that the Realtek (re0) cards have significant
network problems when trying to saturate a network. If you have the
ability try switching to a Intel card (em0) for a lot better
performance, lower interrupts and less CPU usage.

Why interrupts are not handled by more CPUs than one? This is probably
the way the driver was built. It is a single processes which is using
the "big lock" method. This keeps all activity for the drive bound to
a single CPU core.

or One CPU handle interrupts from one card, so I need two NICs?... Two
nics would be a very good idea. You will see better performance a less
IRQ splitting.

Why it is lowered by twice? The CPU load is when the CPU is busy and
can not be used by any other processes. This does _not_ mean that
processing is going on, just that the CPU is unavailable. IRQ's are
like locks and they keep the cpu from being use and hold on to the
cpu. So, irq256 is holding onto the cpu, but not actually processing
any data. This is not very efficient as you can see.

Try changing cards to an Intel variety and use two nics in total; one
for incoming connections and one for outgoing. On the network
performace page we specify the cards we are currently using. Intel
PRO/1000 GT PCI PWLA8391GT can be found on newegg for as little as $31
each.

Hope this helps.

--
   Calomel @ https://calomel.org
   Open Source Research and Reference


On Fri, Nov 18, 2011 at 02:41:15AM -0500, ??????? ??????? wrote:
>Hi.
>
>FreeBSD 9.0-CURRENT FreeBSD 9.0-CURRENT #4: Fri Jun 10 01:30:12 UTC 2011 :/usr/obj/usr/src/sys/PAE_KES  i386
>
>
>I have some quiestions about FreeBSD tunig https://calomel.org/network_performance.html
>
>I have re0 Gigabit Ethernet NIC(NDIS 6.0) (RTL8168/8111/8111c) and core i3 2100
>and two vlans on it: the one for incoming and the other for outgoing packets.
>
>#top -SIHP
>last pid: 14902;  load averages:  1.92,  2.12,  1.96    up 0+17:47:31  19:59:04
>226 processes: 12 running, 197 sleeping, 17 waiting
>CPU 0:  0.6% user,  0.0% nice,  1.2% system, 88.3% interrupt,  9.8% idle
>CPU 1:  1.8% user,  0.0% nice, 29.4% system,  0.0% interrupt, 68.7% idle
>CPU 2:  3.7% user,  0.0% nice, 30.7% system,  0.0% interrupt, 65.6% idle
>CPU 3:  3.1% user,  0.0% nice, 25.8% system,  0.0% interrupt, 71.2% idle
>Mem: 264M Active, 1641M Inact, 272M Wired, 832K Cache, 112M Buf, 1721M Free
>Swap: 4096M Total, 4096M Free
>
>  PID USERNAME   PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
>   12 root       -92    -     0K   152K CPU0    0 354:30 96.78% {irq256: re0}
>   11 root       155 ki31     0K    32K RUN     1 929:16 77.83% {idle: cpu1}
>   11 root       155 ki31     0K    32K RUN     3 922:41 72.95% {idle: cpu3}
>   11 root       155 ki31     0K    32K RUN     2 904:02 71.63% {idle: cpu2}
>   13 root       -16    -     0K    32K CPU3    1  71:11 18.65% {ng_queue1}
>   13 root       -16    -     0K    32K RUN     1  71:10 18.36% {ng_queue3}
>   13 root       -16    -     0K    32K RUN     3  71:18 17.63% {ng_queue0}
>   13 root       -16    -     0K    32K RUN     1  71:11 17.14% {ng_queue2}
>   11 root       155 ki31     0K    32K RUN     0 682:25 10.55% {idle: cpu0}
>55709 root        20    0 13408K  5840K select  2  15:50  1.71% snmpd
>14902 cacti       33    0 11960K  3480K select  1   0:00  1.12% snmpget
>14864 cacti       46    0 11116K  2836K piperd  3   0:00  1.12% perl5.10.1
>14867 root        46    0  9728K  1956K select  3   0:00  1.12% sudo
>
>as you can see irq256 take all CPU0 time and packets that travel
>through router have a lose about 5-10%, CPU100% loaded when trafic
>achive 400Mbit/s and then lower as twice
>
>Now questions
>1. Why interrupts are not handled by more CPUs than one?
>
>2. or One CPU handle interrupts from one card, so I need two NICs?...
>
>3. Why it is lowered by twice?
>
>Thank you.
>
>-- 
>? ?????????,
> ???????                          mailto:kes-kes at yandex.ru