Re: RFC - Work on FreeBSD's Audio Stack

Am 2023-12-14 03:59, schrieb Joseph Mingrone:

> * Deliverables
> 
> Note: By nature of the project, it is possible that the exact details 
> of some deliverables may change. The deliverable list
> mentioned in the proposal is likely to grow if time constraints allow, 
> so consider it a non-exhaustive list.

If you want to integrate some drivers:
https://lists.freebsd.org/pipermail/freebsd-multimedia/2019-June/019719.html
https://lists.freebsd.org/pipermail/freebsd-multimedia/2019-September/019907.html

No idea if they are good quality drivers or not, and if those cards are 
still available, but at least someone tried to support them it may be 
sensible to check if it makes sense to do something with them or not.

> snd hda(4) pin-patching

Great.

> snd uaudio(4) fixes

> Other USB audio bugs include 1) those mentioned in the snd uaudio(4) 
> man page, 2) snd uaudio(4) not passing enough
> information (e.g device name, associated device in /dev, and more) to 
> the OSS API, 3) no explicit list of supported sound
> cards.

Regarding 2) we may want to think about increasing the default verbosity 
of /dev/sndstat...

> MIDI device cloning
> 
> Unlike DSP devices, sound(4) does not allow cloning of MIDI devices, 
> meaning applications cannot open the same MIDI
> device multiple times (e.g when recording more than one track with the 
> same device). This can be verified by tracing
> the dsp clone() routine found in sys/dev/sound/pcm/dsp.c, which is 
> triggered during a dev clone EVENTHANDLER(9)
> event. For example, given a /dev/dsp8.0 device, trying to cat(1) 
> /dev/dsp8.1 would trigger dsp clone() and create it on
> the fly (see FILES section in sound(4) man page). However, doing that 
> with a MIDI device would trigger the handler,
> but result in an error, as MIDI devices do not support cloning. To fix 
> this, the MIDI code will be patched to use the
> snd clone framework used for cloning DSP devices.

The dsp deices are clonable due to the virtualisation and auto-mixing 
into the hardware in the kernel. To my understanding the midi devices 
are not clonable as there is no generic way we could mix several outputs 
from programs into one hardware channel.

> Other kernel improvements
> 
> Other improvements to the kernel code include 1) better-syncing with 
> the 4Front OSSv4 API, 2) addressing open bug
> reports, 3) making optimizations where possible.

1) I mentored a student in the Google summer of code which resulted in 
what we have now in the OSSv4 API. Parts of this work was done as stubs, 
e.g.
#ifdef OSSV4_EXPERIMENT
static int dsp_oss_getlabel(struct pcm_channel *wrch, struct pcm_channel 
*rdch, oss_label_t *label);
static int dsp_oss_setlabel(struct pcm_channel *wrch, struct pcm_channel 
*rdch, oss_label_t *label);
static int dsp_oss_getsong(struct pcm_channel *wrch, struct pcm_channel 
*rdch, oss_longname_t *song);
static int dsp_oss_setsong(struct pcm_channel *wrch, struct pcm_channel 
*rdch, oss_longname_t *song);
static int dsp_oss_setname(struct pcm_channel *wrch, struct pcm_channel 
*rdch, oss_longname_t *name);
#endif
and also dsp_oss_setchnorder().

Maybe you are interested to un-stub them.

> oss(3)
> 
> Following the motivation behind mixer(3), which was to make OSS mixer 
> development more pleasant and easier, I will
> implement an oss(3) library, responsible for handling the audio 
> device’s state. oss(3) will serve as a unified interface for OSS
> audio and MIDI developers and allow for quicker application 
> development, like Digital Audio Workstations, instrument
> effects, audio servers, and general utilities.
> 
> Widely-used existing codebases that can benefit from oss(3) are virtual 
> oss and Mozilla’s cubeb oss audio framework,
> which are also great sources of inspiration for features included in 
> oss(3).

VirtualOSS is now unmaintained (the author died). Maybe we want to adopt 
it into the base?

> audio(8)
> 
> FreeBSD lacks an easy-to-use audio device handling utility similar to 
> mixer(8), but for the device-side of OSS, like
> OpenBSD’s audioctl(8). Such a utility will make use of the new oss(3) 
> library, and offer a cleaner and user-friendlier
> interface, concentrating all useful - but scattered - information 
> currently provided by /dev/sndstat, hw.snd, dev.pcm and
> hw.usb.uaudio, the OSS API into the output of a single application.

The kernel code also contains some XXX comments with some userland 
ideas, e.g. for the buffersize.

> Hot-swapping
> 
> Users of plain OSS, that is, without virtual oss, will have noticed 
> that hot-swapping audio devices (i.e changing the default
> unit, hw.snd.default unit) mid-track does not work and sound keeps 
> coming out of the previous device until the track
> is restarted. This is because applications open(2) the device at the 
> start of playback and close(2) it when the track has
> stopped. virtual oss(8) can create a virtual audio device responsible 
> for routing audio to the appropriate physical audio
> device. The device(s) sound is routed to can be changed at run-time 
> using virtual oss cmd(8).
> 
> This method has the advantage that sound can be re-routed to different 
> devices while applications keep the same /dev/dsp
> open; virtual oss(8) will make any necessary conversions on the fly and 
> redirect audio to the new device without requiring
> a restart of the track/application.
> 
> This functionality will be embedded in either mixer(8) or the new 
> audio(8) program.
> 
> Bluetooth device management utility
> 
> Although not strictly audio-related, I plan to write a (possibly using 
> bsddialog) bluetooth device management utility.
> Setting up bluetooth devices on FreeBSD is still rather complicated and 
> confusing, and involves setting up multiple
> 
> different programs to even pair with a bluetooth device. This tool will 
> detect nearby bluetooth devices, prompt the user
> to choose which one(s) they want to (un-)pair with, and handle 
> configuration automatically.

Someone started with a bluetooth daemon a while ago. It sounds like this 
and your proposal share a common goal. No idea what the state of it is, 
but maybe you want to check it out:
     
https://lists.freebsd.org/archives/freebsd-bluetooth/2022-August/000021.html

> Documentation
> 
> The documentation part of the project includes updating the Wiki, 
> Handbook and Foundation pages to reflect the most
> recent changes and additions. New Handbook sections will be added to 
> showcase the use of oss(3) and mixer(3). with
> additional examples added to /usr/share/examples/sound. Man pages with 
> missing, incomplete or outdated information
> will be addressed.

Some parts of the kernel sound code contain docs written in doxygen 
syntax. A rendered version of it (if there is no error in the daily 
generation for -current) is available at
     https://www.leidinger.net/FreeBSD/dox/dev_sound/html/
and the corresponding TODO markup is rendered at
     
https://www.leidinger.net/FreeBSD/dox/dev_sound/html/dd/da0/todo.html
an example of the function docs (for dsp_oss_syncgroup()) is at
     
https://www.leidinger.net/FreeBSD/dox/dev_sound/html/d5/d3b/dsp_8c.html#a304ece74c26d57a7df3ab5f104a3feff

It would be nice if changes to the kernel maintain/add to the doxygen 
markup.

Bye,
Alexander.

-- 
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org    netchild@FreeBSD.org  : PGP 0x8F31830F9F2772BF