svn commit: r291142 - in head/sys: arm/arm arm64/arm64 mips/mips powerpc/powerpc x86/x86

[Adrian Chadd]

On 22 November 2015 at 20:47, Ian Lepore <ian at freebsd.org> wrote:
> On Sun, 2015-11-22 at 19:22 -0800, Mark Johnston wrote:
>> On Sat, Nov 21, 2015 at 07:55:01PM +0000, Svatopluk Kraus wrote:
>> > Author: skra
>> > Date: Sat Nov 21 19:55:01 2015
>> > New Revision: 291142
>> > URL: https://svnweb.freebsd.org/changeset/base/291142
>> >
>> > Log:
>> >   Fix BUS_DMA_MIN_ALLOC_COMP flag logic. When bus_dmamap_t map is
>> > being
>> >   created for bus_dma_tag_t tag, bounce pages should be allocated
>> >   only if needed.
>> >
>> >   Before the fix, they were allocated always if
>> > BUS_DMA_COULD_BOUNCE flag
>> >   was set but BUS_DMA_MIN_ALLOC_COMP not. As bounce pages are never
>> > freed,
>> >   it could cause memory exhaustion when a lot of such tags together
>> > with
>> >   their maps were created.
>> >
>> >   Note that there could be more maps in one tag by current design.
>> >   However BUS_DMA_MIN_ALLOC_COMP flag is tag's flag. It's set after
>> >   bounce pages are allocated. Thus, they are allocated only for
>> > first
>> >   tag's map which needs them.
>>
>> This appears to cause a hang with one of the re(4) interfaces in my
>> workstation. I can use it to start an ssh session, but it quickly
>> hits a
>> point where it stops transmitting or receiving packets, and I need to
>> reboot the system to recover. Interestingly, the other re(4)
>> interface
>> in this system works fine, but it drives a different card:
>>
>> working:
>> re0 at pci0:3:0:0: class=0x020000 card=0x85051043 chip=0x816810ec
>> rev=0x09 hdr=0x00
>> vendor     = 'Realtek Semiconductor Co., Ltd.'
>> device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet
>> Controller'
>> class      = network
>> subclass   = ethernet
>>
>> non-working:
>> re1 at pci0:5:0:0: class=0x020000 card=0x816910ec chip=0x816910ec
>> rev=0x10 hdr=0x00
>> vendor     = 'Realtek Semiconductor Co., Ltd.'
>> device     = 'RTL8169 PCI Gigabit Ethernet Controller'
>> class      = network
>> subclass   = ethernet
>>
>
> With the old logic (which I'm no great defender of beyond "it worked"),
> with every map added to a tag, some extra bounce pages would be added
> to the tag's bounce zone to account for the new map's needs, up to some
> arbitrary #define'd limit.
>
> With the new logic, it appears that pages will only be added to a
> bounce zone one time for each tag, either when the tag is created or
> when the first map for the tag is created.  After that no more bounce
> pages are ever added no matter how many more maps are created for that
> tag.  So a network driver that creates 1024 maps off a single tag may
> have to bounce every single one of those transfers due to alignment but
> may have only enough resources to bounce one transfer at a time.
>
> More likely it will have enough to do several mappings at a time,
> because usually there's only one bounce zone being used by all tags and
> maps, so extra pages will get added for other tags that will never
> bounce, and they'll get consumed by a driver that does need to bounce.
>
> I would especially expect trouble on ARM where many many mappings need
> to bounce due to alignment (but I haven't actually had any trouble
> since this change went in).

... uhm, shall we revert this then? This seems very risky.



-adrian