re0 fix that works with polling
John-Mark Gurney
gurney_j at resnet.uoregon.edu
Tue Oct 19 01:12:04 PDT 2004
Sean McNeil wrote this message on Mon, Oct 18, 2004 at 23:00 -0700:
> On Mon, 2004-10-18 at 22:33, John-Mark Gurney wrote:
> > Sean McNeil wrote this message on Mon, Oct 18, 2004 at 16:25 -0700:
> > > > Do you have a video that you could send me, that I could do some testing
> > > > with this? (and vls config)? I've been doing my testing on an i386,
> > > > but that shouldn't be different enough to cause problems... (if it is,
> > > > then we need to think about what the re is behaving badly)...
> > >
> > > I have placed 11 megs of the stream in
> > >
> > > http://mcneil.com/~sean/freebsd/stream.mpg
> >
> > damn, is that hdtv quality?? :) looks pretty nice...
>
> Yes, that is low-quality HD at 15mbps. It isn't too bad, though, until
> it gets to a part I don't think was in the short sample I provided.
> Gandalf is attacking the fire demon as they fall. Very blocky.
well, I guess I'm mostly impressed by the high res.. the window at
1x res doesn't even fit on my laptop's display... :)
> > > You can install vls from ports (net/vls) and I run it with
> > >
> > > vls -d udp:224.1.1.1:1234 file:stream.mpg
> > >
> > > I agree that your assesment appears to be accurate in identifying no
> > > packet loss. Are you going through a switch or is this a cross from
> > > machine-machine?
> >
> > The bad news is that it appears that I'm not dropping any packets at
> > all... I'm using vlc on my mac (which is also at gige), and it appears
> > to be getting full data rate.. though things are gittery,but the logs
> > are kinda wierd...
>
> The mac probably doesn't have enough horse power. Typically you need an
> Intel 3GHz processor to get it to play smoothly with software.
yeh, not surprised... my laptop is only 867mhz...
> > now the bad part of the news... if I drop my re0 card over to my 100mbit
> > switch:
> > media: Ethernet autoselect (100baseTX <full-duplex>)
> >
> > Ok, here's a more complete run of what happened:
> > re0 <-> SMC gige switch <-> NetGear gige switch <-> MacOSX laptop (gige)
> > no packet loss observed.. ~1.9megs/sec or 1400pps received at MacOSX
> > laptop
> >
> > re0 <-> summit48 (100mbit) <-> SMC gige switch <-> MacOSX laptop (gige)
> > w/ or w/o netgear switch between SMC and laptop, packet loss is
> > observed, only about 700-900kbyte/sec or 500-600pps
>
> This is very interesting. This is completely opposite to my
> experience. I have
>
> re0 <-> linksys gige <-> hardware decoder or freebsd box just doing
> capture both at 100bt
>
> packet loss. With mods, no loss.
>
> re0 at 100bt <-> linksys 100bt <-> same as above
>
> no packet loss.
>
> Could this be some sort of n-way negotiation issue?
hmmm.. well, just make sure that the machines and the switches all
agree... I couldn't hardcode re0 to 100baset w/ it plugged into the
gige switch (though I didn't try hard coding it, and pulling the plug)..
it always autodetected fine for me...
> > now a bit more information, I did use snmpnetstat on the switch to verify
> > that the switch was receiving (but I forgot to verify that it was sending
> > all packets to the gige port) all packets even w/ the final packet loss..
> >
> > re0 <-> summit48 (100mbit) <-> MacOSX laptop
> > no packet loss observed... similar stats as to the straight gige above..
> >
> > now, I also tried receiving on win2k box, and I received a bit better
> > frame rate.. I don't think my macosx laptop can handle rendering the
> > data stream (or at least vlc doesn't know how to make use of the proper
> > acceleration).. the frame rate was really choppy, and sound not to good..
>
> Yes, that would be expected. Fastest laptop being a 1.5GHz and vlc may
> not be taking advantage of altivec. But then again maybe it is. Never
> tried it on mine.
>
> > Do you have an application that gets good stats from vlc? I did see
> > a few messages in the message log about dropped packets... but I coudln't
> > really figure out how many/much they were, and considering it was udp,
> > I expected some loss...
>
> The best thing I can recommend is to just capture the stream. I have a
> program at
>
> http://www.mcneil.com/~sean/freebsd/stream.c
>
> just compile as cc -o stream stream.c and run as
>
> ./stream 224.1.1.1:1234 >capture.mpg
>
> assuming address:port as before.
>
> I have another program that will compare the streams and report missing
> packets if you find that they appear to be dropping for you.
Yeh, it appears I'm getting the drops here using your stream capture
program... around 16%-20% of the file..
w/ your fix of the timer down to 0x200 I still see about 5% data drop..
and increasing the ring buffer size to 512 (w/o your timer fix) does
see as low as 3% data drop, but I've still seen a 13% drop with it..
with both a ring increase and your fix, I normally saw about 4% data
loss, but I did get one run with 18% loss... now, I didn't run hard
numbers, so these numbers aren't statisticly proven...
The biggest problem I see with your patch is that it appears to
completely disable any type of interrupt moderation... we'd probably
be better off w/ enable tx interrupts than using the timer... with
my tests with your timer fix (just setting it to 0x200), I always saw
double interrupts, one for transmit, and a second for receive... When
I ran the netsend program at 14kpps, I was seeing 27kints!! this was
using like
(note netsend pkt size of 1316 to try to emulate your side)
just for kicks I decided to double the timer w/ a ring buffer of 512,
at 14kpps to 0x1000, and when doing 14kpps I only see 64int/sec w/o any
errors returned (and apparent 100% receive rate) by netsend...
after further examination, I believe I did find a slight logic flaw
with the timer... Every time a packet was delivered to the interface,
the timer was reset... which means it might of been possible to start
the buffers, and with a shorter timer, this avoided the problem...
attached is a patch (with a new sysctl that lets you adjust the timer
on the fly for quicker/easier testing of different values)... It also
prevents prematurely clearing the OACTIVE flag...
see if you get better results w/o having to adjust the timer down..
> > again, most of the stats was gathered with netstat, since this is the
> > easiest way I know to see things happening..
>
> I really do appreciate the attention you are giving this, Jean-Mark.
:) I'm not French.. it's John-Mark.. Considering your name is Sean,
I can see how'd your fingers might make a mistake...
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
-------------- next part --------------
Index: if_re.c
===================================================================
RCS file: /home/ncvs/src/sys/dev/re/if_re.c,v
retrieving revision 1.35
diff -u -r1.35 if_re.c
--- if_re.c 2004/09/28 18:22:24 1.35
+++ if_re.c 2004/10/19 08:09:16
@@ -111,6 +111,7 @@
#include <sys/param.h>
#include <sys/endian.h>
#include <sys/systm.h>
+#include <sys/sysctl.h>
#include <sys/sockio.h>
#include <sys/mbuf.h>
#include <sys/malloc.h>
@@ -156,6 +157,9 @@
#define RE_CSUM_FEATURES (CSUM_IP | CSUM_TCP | CSUM_UDP)
+static int retimerval = -1;
+SYSCTL_INT(_hw, OID_AUTO, re_timer, CTLFLAG_RW, &retimerval, 0, "re tx mod timer value");
+
/*
* Various supported device vendors/types and their names.
*/
@@ -1712,11 +1716,8 @@
/* No changes made to the TX ring, so no flush needed */
- if (idx != sc->rl_ldata.rl_tx_considx) {
+ if (idx != sc->rl_ldata.rl_tx_considx)
sc->rl_ldata.rl_tx_considx = idx;
- ifp->if_flags &= ~IFF_OACTIVE;
- ifp->if_timer = 0;
- }
/*
* If not all descriptors have been released reaped yet,
@@ -1726,6 +1727,12 @@
*/
if (sc->rl_ldata.rl_tx_free != RL_TX_DESC_CNT)
CSR_WRITE_4(sc, RL_TIMERCNT, 1);
+
+ /* only when we are completely done clear the OACTIVE and timer */
+ if (sc->rl_ldata.rl_tx_free == RL_TX_DESC_CNT) {
+ ifp->if_flags &= ~IFF_OACTIVE;
+ ifp->if_timer = 0;
+ }
}
static void
@@ -2008,7 +2015,9 @@
struct rl_softc *sc;
struct mbuf *m_head = NULL;
int idx;
+ int oactive;
+ oactive = ifp->if_flags & IFF_OACTIVE;
sc = ifp->if_softc;
RL_LOCK_ASSERT(sc);
@@ -2058,8 +2067,11 @@
* the value in the TIMERINT register, and then trigger an
* interrupt. Each time we write to the TIMERCNT register,
* the timer count is reset to 0.
+ *
+ * Only reset the timer if we are not currently sending pkts
*/
- CSR_WRITE_4(sc, RL_TIMERCNT, 1);
+ if (!oactive)
+ CSR_WRITE_4(sc, RL_TIMERCNT, 1);
/*
* Set a timeout in case the chip goes out to lunch.
@@ -2216,10 +2228,17 @@
* reloaded on each transmit. This gives us TX interrupt
* moderation, which dramatically improves TX frame rate.
*/
+ if (retimerval == -1) {
+ if (sc->rl_type == RL_8169)
+ retimerval = 0x800;
+ else
+ retimerval = 0x400;
+ }
if (sc->rl_type == RL_8169)
- CSR_WRITE_4(sc, RL_TIMERINT_8169, 0x800);
+ CSR_WRITE_4(sc, RL_TIMERINT_8169, retimerval);
else
- CSR_WRITE_4(sc, RL_TIMERINT, 0x400);
+ CSR_WRITE_4(sc, RL_TIMERINT, retimerval);
+ printf("re timer set to: 0x%x\n", retimerval);
/*
* For 8169 gigE NICs, set the max allowed RX packet
More information about the freebsd-current
mailing list