Provide the tools to pass devices (and the rest) around across Send/Sync boundaries

[MOZGIII]

Devices and Streams aren't Send/Sync.

To pass them around, we can create a thread to bind them all to, and expose channel-based control interface over it to allow passing commands to streams and devices.

This does not have to be baked into the device implementation, but can be a separate utility for users to manually use if they need to.

The reason to include it into cpal is it's that non Send/Sync is a very serious limitation, and it should be solved at the library level, since the underlying design limitation also comes from the library level (i.e. it affects everyone).

[sniperrifle2004]

I don't know. It seems like a bad idea (Send maybe, but Sync?). Stream might be Sync (Might be some implementation caveats, but it's likely doable) but Device is just asking for trouble. Device is already not completely safe (I can start multiple output streams at the same time. It fails miserably, but Device is not preventing me from doing so) and you want to add Sync to that?

EDIT: On top of that it's not easy to find safe semantics that fit all use cases. I would argue for not allowing multiple streams in one direction at all (ie. Failing, but gracefully) but you could argue that it should wait until the device becomes available. Or kick out the running stream. Which should we choose?

EDIT: How would play and pause on passed around streams work? What about the potential effect of old streams on currently running streams? Or even on future streams? Right now those aren't really big problems, but if you start sharing and sending stuff that begins to matter.

[MOZGIII]

What I have in mind is a simple thread holding to a non-Send/Sync Device(s) with a state machine and a set of channels to pass commands in and extract events out. It's what I can already implement currently with existing interfaces, but I figured it would be something useful to have at the lib level.

I wasn't aware it's problematic to have multiple streams in one direction with a single Device. Clearly, it's not by design, but I wonder how it's broken to avoid relying on that - and maybe to land a workaround in place. Could you please link an issue number on this bug for context?

EDIT: How would play and pause on passed around streams work? What about the potential effect of old streams on currently running streams? Or even on future streams? Right now those aren't really big problems, but if you start sharing and sending stuff that begins to matter.

To manage streams, we could send back a stream handle as a result of a creation command. Then, to perform an op on that stream, we pass the handle as part of the control message over the channel.

[sniperrifle2004]

Well it was the first problematic scenario that popped into my head with a Sync device (The full featured Device) so I tried it out in the current situation. Hence there isn't an issue for it yet (As far as I'm aware).

I think the creation command is subject to the same problems as the method is. So it would still require answers to those questions. You are just shifting maintaining the Sync invariants (Which are still not clearly defined) from Device itself to the container thread. That has the advantage that Device doesn't carry the associated costs (So if you don't need it you don't have to pay for it), but you still have the same problems to consider.

EDIT: It's probably less problematic than it sounds. It seemed to fail pretty badly, but that was due to poor error handling. It errors out okay (Though the message is misleading, since it suggest the device is not available, while it is actually busy). That's not exactly the same as failing because a stream is already running, but the result is the same. However there does appear to be some race condition since in debug mode the behaviour changed. It's hard to reproduce but it looks like in some cases the device will (also?) be available to the second stream. I am having too much difficulty reproducing it at the moment. I would like to completely rule out that it was some fluke due to stale build artifacts or something. Though for that to happen twice...

UPDATE: Yes it looks like it was my IDE at fault. Though the line numbers matching up with the breakpoint while the source has changed is unexpected. It is the only thing that explains why right after adding the second stream to the code, it works until I Run instead of Debug the new version. After that it begins failing consistently. That's a nuisance, but I am relieved nonetheless.

[sniperrifle2004]

Ah I think I understand what is going on. Your device is capable of mixing and that's why for you it seems perfectly normal for more streams to be running on the same device. Now I was addressing my card directly and thus it failed, since my sound card doesn't have a mixer. Now there are more potential problems I can think of with concurrent streams based on the device interface (The device would also need to be able to handle different configs at the same time). So for me it wasn't at all obvious that multiple streams were supported by design. To be honest it still isn't. At least not inherently supported.

This does answer the question how you want multiple streams to be handled. You want them to be mixed. Unfortunately that is not a promise the device your thread is holding is guaranteed to deliver. So maybe the feature you are looking for is a Mixer device? Your thread does seem to be fulfilling that role. It could even be Sync.

[MOZGIII]

Ah I think I understand what is going on. Your device is capable of mixing and that's why for you it seems perfectly normal for more streams to be running on the same device. Now I was addressing my card directly and thus it failed, since my sound card doesn't have a mixer. Now there are more potential problems I can think of with concurrent streams based on the device interface (The device would also need to be able to handle different configs at the same time). So for me it wasn't at all obvious that multiple streams were supported by design. To be honest it still isn't. At least not inherently supported.

I'm not targeting a specific sound card, but rather building a portable cross-platform app. It's generally supported across all platforms to have multiple streams (even on Linux PulseAudio emulates ALSA interface and does the mixing for the system).

But then, I don't care at all about having multiple streams in two directions, I'll only be using one capture and one playback stream. My problem is purely that I want to be able to synchronize access to cpal from multiple threads - ideally in an async fashion.

I do some mixing internally, but that's totally unrelated to the cpal layer. Also, the callback interfaces are already fine - I'm having no issues cause I have Send ringbuffers with disjoint writer and reader ends. My problem is just that when I do the device initialization and format negotiation - I already need to cross Send boundaries a couple of times before I can spawn a stream. Thus, I don't care as much about the Sync either.

It used to work fine with an event loop, because then the Device didn't have those limitations.

[sniperrifle2004]

I'm not targeting a specific sound card, but rather building a portable cross-platform app. It's generally supported across all platforms to have multiple streams (even on Linux PulseAudio emulates ALSA interface and does the mixing for the system).

But I was, which is why it failed. The "default" card is often (But not always. Most of the time. Probably) some software over hardware that does the mixing (Also not always. Even if it is software). When I changed it back to my "default" device it worked. Now I had never heard of the mixing of said device being relied on by an application rather than that it just provided the mixing between different applications before. I'm not saying Device can't possibly support multiple output streams. Just that Device doesn't in and of itself provide it. In order for it to do that the underlying device must do the mixing. And possibly resampling. So it would be a bad idea for cpal to rely on it and make Device sync. Whether as a thread or by just marking it. Now a Mixer device could add mixer functionality to a Device. Sort of like your thread except that still relies on the Device to do the mixing. If the device doesn't do it, it will not work. From my point of view any Device could be a physical device or another device that simply doesn't support mixing. And if it supported mixing, it might not also support different sample rates for the different streams. Which is why, when I look at device, I don't expect any more than one output stream to work all the time. If it were a dedicated recording device that wouldn't even be possible obviously. It's why I'm sort of worried that Device has an interface that suggest half- or full-duplex capabilities. If the device didn't have that functionality it would probably fail, but the user might not understand that. Now the default input and default output device promise something, but the entries in Devices not so much.

[MOZGIII]

This is going slightly off-topic. I suggest you create another issue because having a wrapper Device that could do mixing and resampling would be useful too. I need a simpler thing though.

[sniperrifle2004]

I don't think you do. Once you hit a Device that doesn't do mixing your simpler thing will not solve your problem any more and your application will not work. Granted that might be unlikely, but in the broad cpal sense that unlikely is a problem.

[MOZGIII]

I don't want cpal to do mixing for my current use case. I do have mixing/resampling/transcoding facilities, but I want them to be separate from cpal and entirely under my control, rather than the library control.

I need a convenient way to cross Send borders. That's it.

I use cpal as just one of the audio backends. I really need it just do it's job to interface with the devices, and that's it. The rest of the logic is additional layers atop.

[sniperrifle2004]

Well the github outage makes this a bit awkward, since I wanted to immediately withdraw my last comment. I rushed it out to soon 😅.

I sincerely and deeply apologize. I completely misunderstood some of your comments. I don't think that sharing a resource that could potentially inherently be, well, not effectively shared (aside possibly from an input/output perspective) is a good idea, but it's not unsafe to do so, provided the resource is well guarded. I mean a thread could be disappointed and not acquire a stream or suddenly loose its stream (Though that would be unlikely) or something, but it won't be unsafe.