DWC OTG TX path optimisation for 11-current

[Hans Petter Selasky]

On 08/12/15 20:16, Svatopluk Kraus wrote:
> On Wed, Aug 12, 2015 at 3:35 PM, Hans Petter Selasky <hps at selasky.org> wrote:
>> On 07/31/15 17:11, Svatopluk Kraus wrote:
>>>
>>> Note that I got about 24000 interrupts per a second during buildworld
>>> too. If I remember it correctly, it was about 4000 before r285935.
>>> When the trigger is pulled, the count is changing (in range of 4000 to
>>> 21000). It looks like something is taking more time (interrupt
>>> servicing probably). According to "systat -v", disk is going to be
>>> 100% (and more) busy very often for example.
>>
>>
>> Hi,
>>
>> Finally my RPI-2 arrived :-)
>
>
> It's great! ;) I can do some debugging but it's hard to go deeper
> without usb controller specification.
>
>
>>
>> After r285935 we are polling for data more frequently, so the IRQ rate will
>> go up. 3x 8000 IRQs/second would be normal to query the device every 125us,
>> which the EHCI does.
>>
>> Can you give me some more information about your setup, so that I can
>> reproduce it?
>
> I use in-tree RPI2 configuration but with INVARIANTS and
> INVARIANT_SUPPORT options disabled. This is crucial as it's hard to
> trigger it with these options enabled.
>
> I use usb disk with three partitions:
> First one is mounted as a root.
> Second one is used for swapping.
> The third one is not mounted.
>
> The disk is connected thru usb hub.
>
> My fstab is empty.
> My kernel is loaded from SD card.
> I use UART as console.
>
> root at raspberry-pi:~ # usbconfig
> ugen0.1: <OTG Root HUB DWCOTG> at usbus0, cfg=0 md=HOST spd=HIGH
> (480Mbps) pwr=SAVE (0mA)
> ugen0.2: <product 0x9514 vendor 0x0424> at usbus0, cfg=0 md=HOST
> spd=HIGH (480Mbps) pwr=SAVE (2mA)
> ugen0.3: <product 0xec00 vendor 0x0424> at usbus0, cfg=0 md=HOST
> spd=HIGH (480Mbps) pwr=ON (2mA)
> ugen0.4: <USB2.0 Hub VIA Labs, Inc.> at usbus0, cfg=0 md=HOST spd=HIGH
> (480Mbps) pwr=SAVE (0mA)
> ugen0.5: <USB2.0 Hub VIA Labs, Inc.> at usbus0, cfg=0 md=HOST spd=HIGH
> (480Mbps) pwr=SAVE (0mA)
> ugen0.6: <HD710 ADATA> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (0mA)
>
>
> FreeBSD source tree is on /usr/src and I run native buildworld : "make
> -j6 buildworld"
> I run the buildworld on ssh to keep my console "clean and free" for
> debugging. The trigger is pulled in about 25 minutes, however it's
> more expessive with more time elapsed.
>
> So far it looks that it happens during "stage 2.1: cleaning up the
> object tree" when a lot of small and different disk request are done.
>
> I have tried to get big picture of the problem with KTR option
> enabled, but it's not possible as it's not triggered when all KTR
> classes are compiled in. What works is the following configuration:
>
> options  KTR
> options  KTR_ENTRIES=65536
> options  KTR_MASK=(KTR_SPARE3)
> options  KTR_COMPILE=(KTR_SPARE3)
>
> However, even using of KTR_SPARE3 class for debugging is limited
> considering the trigger.
>
> I will send you more info about my investigation tomorow.
>
> Svata
>

Hi,

Here are some more clues. I've found a situation where if one USB class 
driver gets too busy, the others gets starved. I have a patch in the 
works to implement a round robin mechanism for this.

In the meanwhile, maybe you can try to:

cpuset -l 1-3 -p XXX

Where XXX is pid for the usbus0 processes, typically 13 with my RPI-2.

The DWC OTG interrupts on CPU #0 and if the usbus0 processes are running 
there too, it is suboptimal.

--HPS