Re: RFC - Work on FreeBSD's Audio Stack

On Wed, Dec 13, 2023 at 10:59:13PM -0400, Joseph Mingrone wrote:
> Re-sending the text because the original contained a few errors, the worst being broken URLs from a PDF to text conversion.
> 
> Hello,
> 
> The FreeBSD Foundation will be contracting out work on FreeBSD's audio stack.  If you have any comments about the work, please share them before next Wednesday, December 20.
> 
> Joe
> 
> ===
> 
> * Executive Summary
> 
> The FreeBSD audio stack is one of those fields that does not attract the same attention and development as others do,
> since it has been left largely unmaintained, and, although high in quality, there is still room for improvement - from lack
> of audio development frameworks, to missing userland utilities and kernel driver-related bugs. This project is meant to
> touch on all those areas, and as such, will be more of a general improvement project, than an implementation of a specific
> feature.
> 
> * Project Description
> 
> The end goal of the project is to provide FreeBSD audio developers with useful tools and frameworks to make sound
> development on FreeBSD easier, without having to resort to custom solutions for everything.
> 
> On the user side, FreeBSD will ship with a more stable audio stack and a larger collection of userland programs, making
> for a more coherent ecosystem in general. OSS generally does not come with many native tools except mixer(8), and users
> have to resort to using tools from multiple different audio systems (ALSA, PulseAudio, sndio) to compensate. Additionally,
> I think the introduction of new development frameworks will encourage more native audio development, which is going to
> be especially beneficial for people - me included - who want to use FreeBSD for music production.
> 
> * Deliverables
> 
> Note: By nature of the project, it is possible that the exact details of some deliverables may change. The deliverable list
> mentioned in the proposal is likely to grow if time constraints allow, so consider it a non-exhaustive list.
> 
> snd hda(4) pin-patching
> 
> Regarding the stability of the audio stack, the project will address the pin-patching issue present in the snd hda(4) (Intel
> High Definition Audio) driver. Essentially, some laptop models have non-standardized mappings for the headphone and
> speaker jack pins, which results in absence of audio from the headphones, until a patch is manually applied in snd hda(4)
> to correctly map the headphone and speaker pins for that model so that the same audio stream is output from both the
> speakers and headphones when they are plugged in.
> 
> The initial strategy to address the issue will be to see if it is possible to do the patching automatically by figuring out
> what the Speaker pin’s as number is (see dev.hdac.<n>.pindump=1) and "forcibly" assign the same value to the headphone
> jack pin as well. The following solutions follow the same logic:
> 
>    - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273809
>    - https://i-bsd.com/freebsd-wireless-8265/
> 
> OpenBSD’s azalia(4) (their version of the HDA driver) will also serve as a point of reference, even though it too contains
> a kind of manual patching mechanism.
> 
> snd uaudio(4) fixes
> 
> The project will also address bugs in the USB audio driver, snd uaudio(4), which I have been able to reproduce using my
> Focusrite Scarlett USB sound card, with the most prominent and consistent one being noise produced during the first 12
> seconds of playback and when moving along a track/video. If the user tries to move forward multiple times in a short time
> window, the audio device most of the time becomes unusable (i.e no audio) and it has to be replugged. Though this issue
> is largely bypassed if audio is routed to the USB device through virtual oss, this is still a bug that needs to be addressed.
> 
> Related bug reports include:
> 
>    - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257082
>    - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194527
> 
> Initially I am going to test whether the open() and read() routines cause this issue, as DTrace suggests that this happens
> around the same time open(2) or the first read(2) is called. As mentioned in the previous paragraph, virtual oss partially
> fixes this issue, so I would like to investigate and understand why, and maybe find the root cause that way. Another
> source of information will be Linux’s Scarlett and USB audio drivers which, as far as I know, do not have this issue.
> 
> Other USB audio bugs include 1) those mentioned in the snd uaudio(4) man page, 2) snd uaudio(4) not passing enough
> information (e.g device name, associated device in /dev, and more) to the OSS API, 3) no explicit list of supported sound
> cards.

Another issue which is painful with uaudio, more and more laptops have uaudio
devices by default instead of hda, and uaudio is not able to suspend/resume if
something is consuming it.

This is the only hotpluggable device we have for snd card, when one is going to
suspend, the device is being detached not suspended (because usb) and the
decatch procedure goes through the pcm_unregister function which loops forever
hoping the application using the device will die, which never happens, so
suspend ends up in an infinite loop in the kernel.

Note the same happen if you disconnect any uaudio device while an application is
using it.

Best regards,
Bapt