[Bug 277476] amdgpu/drm-kmod periodic hangs due to phys contig allocations
- Reply: bugzilla-noreply_a_freebsd.org: "[Bug 277476] amdgpu/drm-kmod periodic hangs due to phys contig allocations"
- Reply: bugzilla-noreply_a_freebsd.org: "[Bug 277476] graphics/drm-515-kmod: amdgpu periodic hangs due to phys contig allocations"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Mon, 04 Mar 2024 14:12:20 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277476
Bug ID: 277476
Summary: amdgpu/drm-kmod periodic hangs due to phys contig
allocations
Product: Ports & Packages
Version: Latest
Hardware: Any
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: Individual Port(s)
Assignee: ports-bugs@FreeBSD.org
Reporter: jeffpc@josefsipek.net
Two weeks ago I replaced an ancient nvidia graphics card with an AMD RX580 card
to run open source drivers. Everything works fine most of the time, but
occasionally the system hangs for a few seconds (5-10, usually). The longer
the system has been up, the worse it gets.
Digging into it a bit, it is because userspace (it always looks like X) does an
ioctl into drm which then tries to allocate a large-ish piece of physically
contiguous memory. This explains why it gets worse as uptime increases (free
physical memory fragmentation) and when running firefox (the most memory hungry
application I use). I know nothing about graphics cards, the software stack
supporting them, or the linux kernel API compatibility layer, but clearly it'd
be beneficial if amdgpu/drm/whatever could make use of *virtually* contiguous
pages or some kind of allocation caching/reuse to avoid repeatedly asking the
vm code for physically contiguous ranges.
To conclude the above, I did a handful of dtrace-based experiments.
While one of the "temporary hangs" was happening, the following was the most
common (non-idle) profiler stack:
# dtrace -n 'profile-97{@[stack()]=count()}'
...
kernel`vm_phys_alloc_contig+0x11d
kernel`linux_alloc_pages+0x8f
ttm.ko`ttm_pool_alloc+0x2cb
ttm.ko`ttm_tt_populate+0xc5
ttm.ko`ttm_bo_handle_move_mem+0xc3
ttm.ko`ttm_bo_validate+0xb4
ttm.ko`ttm_bo_init_reserved+0x199
amdgpu.ko`amdgpu_bo_create+0x1eb
amdgpu.ko`amdgpu_bo_create_user+0x21
amdgpu.ko`amdgpu_gem_create_ioctl+0x1e2
drm.ko`drm_ioctl_kernel+0xc6
drm.ko`drm_ioctl+0x2b5
kernel`linux_file_ioctl+0x312
kernel`kern_ioctl+0x255
kernel`sys_ioctl+0x123
kernel`amd64_syscall+0x109
kernel`0xffffffff80fe43eb
The latency of vm_phys_alloc_contig (entry to return) is bimodal - with
latencies in the single digit *milli*seconds during the "temporary hangs":
# dtrace -n 'fbt::vm_phys_alloc_contig:entry{self->ts=timestamp}' -n
'fbt::vm_phys_alloc_contig:return/self->ts/{this->delta=timestamp-self->ts;
@=quantize(this->delta);}' -n 'tick-1sec{printa(@)}'
...
value ------------- Distribution ------------- count
256 | 0
512 |@ 2606
1024 |@@@@@@@@@ 18207
2048 |@ 2534
4096 | 894
8192 | 34
16384 | 78
32768 | 58
65536 | 219
131072 | 306
262144 | 310
524288 | 735
1048576 | 174
2097152 |@@ 4364
4194304 |@@@@@@@@@@@@@@@@@@@@@@@@ 47475
8388608 |@ 1546
16777216 | 2
33554432 | 0
The number of pages being allocated:
# dtrace -n 'fbt::vm_phys_alloc_contig:entry/arg1>1/{@=quantize(arg1)}' -n
'tick-1sec{printa(@)}'
...
value ------------- Distribution ------------- count
1 | 0
2 |@@@ 15
4 |@ 7
8 |@@@ 16
16 |@@ 10
32 |@@ 10
64 |@ 7
128 |@@@ 12
256 |@@@ 12
512 |@@@@@@@@@@@@@@ 68
1024 |@@@@@@@ 32
2048 | 0
I did a few more dtrace experiments, but they all point to the same thing - a
drm/amdgpu related ioctl wants 4MB of physically contiguous memory often enough
to become a headache. 4MB isn't too much given than the system has 32GB of
RAM, but physically contiguous takes a while to fulfill sometimes.
The card:
vgapci0@pci0:1:0:0: class=0x030000 rev=0xe7 hdr=0x00 vendor=0x1002
device=0x67df subvendor=0x1da2 subdevice=0xe353
vendor = 'Advanced Micro Devices, Inc. [AMD/ATI]'
device = 'Ellesmere [Radeon RX 470/480/570/570X/580/580X/590]'
class = display
subclass = VGA
$ pkg info|grep -i amd
gpu-firmware-amd-kmod-aldebaran-20230625 Firmware modules for aldebaran AMD
GPUs
gpu-firmware-amd-kmod-arcturus-20230625 Firmware modules for arcturus AMD GPUs
gpu-firmware-amd-kmod-banks-20230625 Firmware modules for banks AMD GPUs
gpu-firmware-amd-kmod-beige-goby-20230625 Firmware modules for beige_goby AMD
GPUs
gpu-firmware-amd-kmod-bonaire-20230625 Firmware modules for bonaire AMD GPUs
gpu-firmware-amd-kmod-carrizo-20230625 Firmware modules for carrizo AMD GPUs
gpu-firmware-amd-kmod-cyan-skillfish2-20230625 Firmware modules for
cyan_skillfish2 AMD GPUs
gpu-firmware-amd-kmod-dimgrey-cavefish-20230625 Firmware modules for
dimgrey_cavefish AMD GPUs
gpu-firmware-amd-kmod-fiji-20230625 Firmware modules for fiji AMD GPUs
gpu-firmware-amd-kmod-green-sardine-20230625 Firmware modules for green_sardine
AMD GPUs
gpu-firmware-amd-kmod-hainan-20230625 Firmware modules for hainan AMD GPUs
gpu-firmware-amd-kmod-hawaii-20230625 Firmware modules for hawaii AMD GPUs
gpu-firmware-amd-kmod-kabini-20230625 Firmware modules for kabini AMD GPUs
gpu-firmware-amd-kmod-kaveri-20230625 Firmware modules for kaveri AMD GPUs
gpu-firmware-amd-kmod-mullins-20230625 Firmware modules for mullins AMD GPUs
gpu-firmware-amd-kmod-navi10-20230625 Firmware modules for navi10 AMD GPUs
gpu-firmware-amd-kmod-navi12-20230625 Firmware modules for navi12 AMD GPUs
gpu-firmware-amd-kmod-navi14-20230625 Firmware modules for navi14 AMD GPUs
gpu-firmware-amd-kmod-navy-flounder-20230625 Firmware modules for navy_flounder
AMD GPUs
gpu-firmware-amd-kmod-oland-20230625 Firmware modules for oland AMD GPUs
gpu-firmware-amd-kmod-picasso-20230625 Firmware modules for picasso AMD GPUs
gpu-firmware-amd-kmod-pitcairn-20230625 Firmware modules for pitcairn AMD GPUs
gpu-firmware-amd-kmod-polaris10-20230625 Firmware modules for polaris10 AMD
GPUs
gpu-firmware-amd-kmod-polaris11-20230625 Firmware modules for polaris11 AMD
GPUs
gpu-firmware-amd-kmod-polaris12-20230625 Firmware modules for polaris12 AMD
GPUs
gpu-firmware-amd-kmod-raven-20230625 Firmware modules for raven AMD GPUs
gpu-firmware-amd-kmod-raven2-20230625 Firmware modules for raven2 AMD GPUs
gpu-firmware-amd-kmod-renoir-20230625 Firmware modules for renoir AMD GPUs
gpu-firmware-amd-kmod-si58-20230625 Firmware modules for si58 AMD GPUs
gpu-firmware-amd-kmod-sienna-cichlid-20230625 Firmware modules for
sienna_cichlid AMD GPUs
gpu-firmware-amd-kmod-stoney-20230625 Firmware modules for stoney AMD GPUs
gpu-firmware-amd-kmod-tahiti-20230625 Firmware modules for tahiti AMD GPUs
gpu-firmware-amd-kmod-tonga-20230625 Firmware modules for tonga AMD GPUs
gpu-firmware-amd-kmod-topaz-20230625 Firmware modules for topaz AMD GPUs
gpu-firmware-amd-kmod-vangogh-20230625 Firmware modules for vangogh AMD GPUs
gpu-firmware-amd-kmod-vega10-20230625 Firmware modules for vega10 AMD GPUs
gpu-firmware-amd-kmod-vega12-20230625 Firmware modules for vega12 AMD GPUs
gpu-firmware-amd-kmod-vega20-20230625 Firmware modules for vega20 AMD GPUs
gpu-firmware-amd-kmod-vegam-20230625 Firmware modules for vegam AMD GPUs
gpu-firmware-amd-kmod-verde-20230625 Firmware modules for verde AMD GPUs
gpu-firmware-amd-kmod-yellow-carp-20230625 Firmware modules for yellow_carp AMD
GPUs
suitesparse-amd-3.3.0 Symmetric approximate minimum degree
suitesparse-camd-3.3.0 Symmetric approximate minimum degree
suitesparse-ccolamd-3.3.0 Constrained column approximate minimum degree
ordering
suitesparse-colamd-3.3.0 Column approximate minimum degree ordering
algorithm
webcamd-5.17.1.2_1 Port of Linux USB webcam and DVB drivers into
userspace
xf86-video-amdgpu-22.0.0_1 X.Org amdgpu display driver
$ pkg info|grep -i drm
drm-515-kmod-5.15.118_3 DRM drivers modules
drm-kmod-20220907_1 Metaport of DRM modules for the linuxkpi-based
KMS components
gpu-firmware-kmod-20230210_1,1 Firmware modules for the drm-kmod drivers
libdrm-2.4.120_1,1 Direct Rendering Manager library and headers
--
You are receiving this mail because:
You are the assignee for the bug.