Use fundsp network to play up to 20 samples at once #35

goodhoko · 2023-04-26T14:40:01Z

@bschwind very dirty example, but it works! You can focus only on the last commit which builds on top of #33. The rest is your branch but rebased on main as I needed tick_probe for testing this out.

The main caveat is that the mixer is built statically. It's type is something like

An<Binop<f64, FrameAdd<UInt<UTerm, B1>, f64>, Binop<f64, FrameAdd<UInt<UTerm, B1>, f64>, Binop<f64, FrameAdd<..., ...>, ..., ...>, ...>, ...>>

Not only such code is ugly and fragile (number of + pass() needs to be kept in sync with NUM_SLOTS) but it also seems that compilation time grows very quickly with the number of slots. (rust-lang/rust#99188 may be to blame)

I'd hope that we could adapt this to using AudioUnits instead of AudioNodes to construct the mixer, but I haven't yet figured out how.

goodhoko · 2023-04-26T14:47:55Z

@strohel the good news is that this overcomes the buffer resolution problem.

When I tried this with 100 slots, I could run tick_probe up to about 200 Hz and the resulting sound's pitch was following nicely.

strohel

Tried running locally, it indeed works.

As expected, when I go beyond tick_probe 20, (to i.e. 50, 100, 200) I can hear how the sounds are cut off (except the last samples when I stop the probe), and it gets gradually quieter (because I guess the beginning of the click sound is rather quiet).

So I fear the 20 parallel sounds would be rather limiting, but I understand going beyond that may be stretching fundsps design.

I'll also try experimenting with driving cpal directly (but still probably using rodio for processing the samples).

strohel · 2023-04-26T16:28:39Z

crates/composer/src/sound.rs

+        // Create a node with 20 inputs that are mixed into one output
+        let mixer = pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass()
+            + pass();


Heh, I guess this cannot use a for loop because this uses some kind of "incremental" generic type? Might be a candidate for a (declarative) macro if we choose to go this way and polish the code.

Yeah, this definitely would benefit from a macro. Not sure it's possible to do with declarative macros alone, though (as in; not sure it's possible to expand a number into a variable number of lines, though I may be forgetting how! the closest I've done is matrix expansion).

Pulling proc macros into this seems too heavyweight, but then I start to wonder if a crate that lets you write code as crazy as this is a good idea 🤔.

@PabloMansanet

…lbacks This is a form of alternative to #35 to resolve #19 (comment) Note that this should resolve the specific problem, but we still may want to use `fundsp` (somehow) to synthetize and modify sound. We still use `rodio` for audio file decoding and processing, but we drive the `cpal` audio callback directly (using code adapted from `rodio` itself). That allows us to run custom code during the callback, which we use to offset samples tasked to play (giving all samples the same delay of one period, rather than variable delay based on how close next to the next callback they fire). <details><summary>See example print of ns_since_last_callback when we play 1000 samples a second</summary> The period length is apparently ~42 ms. ``` ns_since_last_callback: 38695172. ns_since_last_callback: 39784768. ns_since_last_callback: 40887270. ns_since_last_callback: 41945052. ns_since_last_callback: 368139. ns_since_last_callback: 1477218. ns_since_last_callback: 2445557. ns_since_last_callback: 3573154. ns_since_last_callback: 4616437. ns_since_last_callback: 5675630. ns_since_last_callback: 6764239. ns_since_last_callback: 7835255. ns_since_last_callback: 8925809. ns_since_last_callback: 9995435. ns_since_last_callback: 11159180. ns_since_last_callback: 12234090. ns_since_last_callback: 13312871. ns_since_last_callback: 14392252. ns_since_last_callback: 15460542. ns_since_last_callback: 16545115. ns_since_last_callback: 17564547. ns_since_last_callback: 18582770. ns_since_last_callback: 19640031. ns_since_last_callback: 20683684. ns_since_last_callback: 21803771. ns_since_last_callback: 22879984. ns_since_last_callback: 23956644. ns_since_last_callback: 25022856. ns_since_last_callback: 26101435. ns_since_last_callback: 27179575. ns_since_last_callback: 28249203. ns_since_last_callback: 29342603. ns_since_last_callback: 30412447. ns_since_last_callback: 31495496. ns_since_last_callback: 32519447. ns_since_last_callback: 33580415. ns_since_last_callback: 34606142. ns_since_last_callback: 35697450. ns_since_last_callback: 36791844. ns_since_last_callback: 37869512. ns_since_last_callback: 38943775. ns_since_last_callback: 40022488. ns_since_last_callback: 41100993. ns_since_last_callback: 42180112. ns_since_last_callback: 481551. ns_since_last_callback: 1608677. ns_since_last_callback: 2667820. ns_since_last_callback: 3745600. ns_since_last_callback: 4785690. ``` </details> With this I'm able to got from `tick_probe 1` all the way to `tick_probe 4000` (!) and still hear difference. [1] [1] However, there seem to be inefficiencies somewhere that result in e.g. `composer` saying `Received 860 events (3440 bytes) in last 1.00s.` when we call `tick_probe 1000`. May be naive timing code in `tick_probe` itself (@PabloMansanet?).

PabloMansanet

Nice work with the example!

I'm still a bit nervous about the weird code that fundsp makes us write, but since this is the experimental phase of the project, I think it's OK.

@PabloMansanet

…lbacks This is a form of alternative to #35 to resolve #19 (comment) Note that this should resolve the specific problem, but we still may want to use `fundsp` (somehow) to synthetize and modify sound. We still use `rodio` for audio file decoding and processing, but we drive the `cpal` audio callback directly (using code adapted from `rodio` itself). That allows us to run custom code during the callback, which we use to offset samples tasked to play (giving all samples the same delay of one period, rather than variable delay based on how close next to the next callback they fire). <details><summary>See example print of ns_since_last_callback when we play 1000 samples a second</summary> The period length is apparently ~42 ms. ``` ns_since_last_callback: 38695172. ns_since_last_callback: 39784768. ns_since_last_callback: 40887270. ns_since_last_callback: 41945052. ns_since_last_callback: 368139. ns_since_last_callback: 1477218. ns_since_last_callback: 2445557. ns_since_last_callback: 3573154. ns_since_last_callback: 4616437. ns_since_last_callback: 5675630. ns_since_last_callback: 6764239. ns_since_last_callback: 7835255. ns_since_last_callback: 8925809. ns_since_last_callback: 9995435. ns_since_last_callback: 11159180. ns_since_last_callback: 12234090. ns_since_last_callback: 13312871. ns_since_last_callback: 14392252. ns_since_last_callback: 15460542. ns_since_last_callback: 16545115. ns_since_last_callback: 17564547. ns_since_last_callback: 18582770. ns_since_last_callback: 19640031. ns_since_last_callback: 20683684. ns_since_last_callback: 21803771. ns_since_last_callback: 22879984. ns_since_last_callback: 23956644. ns_since_last_callback: 25022856. ns_since_last_callback: 26101435. ns_since_last_callback: 27179575. ns_since_last_callback: 28249203. ns_since_last_callback: 29342603. ns_since_last_callback: 30412447. ns_since_last_callback: 31495496. ns_since_last_callback: 32519447. ns_since_last_callback: 33580415. ns_since_last_callback: 34606142. ns_since_last_callback: 35697450. ns_since_last_callback: 36791844. ns_since_last_callback: 37869512. ns_since_last_callback: 38943775. ns_since_last_callback: 40022488. ns_since_last_callback: 41100993. ns_since_last_callback: 42180112. ns_since_last_callback: 481551. ns_since_last_callback: 1608677. ns_since_last_callback: 2667820. ns_since_last_callback: 3745600. ns_since_last_callback: 4785690. ``` </details> With this I'm able to got from `tick_probe 1` all the way to `tick_probe 4000` (!) and still hear difference. [1] [1] However, there seem to be inefficiencies somewhere that result in e.g. `composer` saying `Received 860 events (3440 bytes) in last 1.00s.` when we call `tick_probe 1000`. May be naive timing code in `tick_probe` itself (@PabloMansanet?).

@PabloMansanet

…lbacks This is a form of alternative to #35 to resolve #19 (comment) Note that this should resolve the specific problem, but we still may want to use `fundsp` (somehow) to synthetize and modify sound. We still use `rodio` for audio file decoding and processing, but we drive the `cpal` audio callback directly (using code adapted from `rodio` itself). That allows us to run custom code during the callback, which we use to offset samples tasked to play (giving all samples the same delay of one period, rather than variable delay based on how close next to the next callback they fire). <details><summary>See example print of ns_since_last_callback when we play 1000 samples a second</summary> The period length is apparently ~42 ms. ``` ns_since_last_callback: 38695172. ns_since_last_callback: 39784768. ns_since_last_callback: 40887270. ns_since_last_callback: 41945052. ns_since_last_callback: 368139. ns_since_last_callback: 1477218. ns_since_last_callback: 2445557. ns_since_last_callback: 3573154. ns_since_last_callback: 4616437. ns_since_last_callback: 5675630. ns_since_last_callback: 6764239. ns_since_last_callback: 7835255. ns_since_last_callback: 8925809. ns_since_last_callback: 9995435. ns_since_last_callback: 11159180. ns_since_last_callback: 12234090. ns_since_last_callback: 13312871. ns_since_last_callback: 14392252. ns_since_last_callback: 15460542. ns_since_last_callback: 16545115. ns_since_last_callback: 17564547. ns_since_last_callback: 18582770. ns_since_last_callback: 19640031. ns_since_last_callback: 20683684. ns_since_last_callback: 21803771. ns_since_last_callback: 22879984. ns_since_last_callback: 23956644. ns_since_last_callback: 25022856. ns_since_last_callback: 26101435. ns_since_last_callback: 27179575. ns_since_last_callback: 28249203. ns_since_last_callback: 29342603. ns_since_last_callback: 30412447. ns_since_last_callback: 31495496. ns_since_last_callback: 32519447. ns_since_last_callback: 33580415. ns_since_last_callback: 34606142. ns_since_last_callback: 35697450. ns_since_last_callback: 36791844. ns_since_last_callback: 37869512. ns_since_last_callback: 38943775. ns_since_last_callback: 40022488. ns_since_last_callback: 41100993. ns_since_last_callback: 42180112. ns_since_last_callback: 481551. ns_since_last_callback: 1608677. ns_since_last_callback: 2667820. ns_since_last_callback: 3745600. ns_since_last_callback: 4785690. ``` </details> With this I'm able to got from `tick_probe 1` all the way to `tick_probe 4000` (!) and still hear difference. [1] [1] However, there seem to be inefficiencies somewhere that result in e.g. `composer` saying `Received 860 events (3440 bytes) in last 1.00s.` when we call `tick_probe 1000`. May be naive timing code in `tick_probe` itself (@PabloMansanet?).

goodhoko · 2024-09-15T19:37:23Z

Not likely to pursue this approach further. Closing.

bschwind and others added 6 commits April 26, 2023 16:05

Add a simple example of using fundsp

a31a14c

Rename c to dsp_graph

afb7df7

Play around with more dsp functions

21239e2

Split the DSP graph into a frontend and backend

44dd826

Add MacOS .DS_Store files to .gitignore

c5552cd

Use fundsp network to play up to 20 samples at once

a252b47

goodhoko requested a review from bschwind April 26, 2023 14:41

strohel reviewed Apr 26, 2023

View reviewed changes

strohel mentioned this pull request Apr 26, 2023

Drive audio directly using cpal, position sounds precisely within callbacks #36

Merged

PabloMansanet approved these changes Apr 27, 2023

View reviewed changes

goodhoko closed this Sep 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use fundsp network to play up to 20 samples at once #35

Use fundsp network to play up to 20 samples at once #35

goodhoko commented Apr 26, 2023 •

edited

Loading

goodhoko commented Apr 26, 2023

strohel left a comment •

edited

Loading

strohel Apr 26, 2023

PabloMansanet Apr 27, 2023

PabloMansanet left a comment

goodhoko commented Sep 15, 2024

Use fundsp network to play up to 20 samples at once #35

Use fundsp network to play up to 20 samples at once #35

Conversation

goodhoko commented Apr 26, 2023 • edited Loading

goodhoko commented Apr 26, 2023

strohel left a comment • edited Loading

Choose a reason for hiding this comment

strohel Apr 26, 2023

Choose a reason for hiding this comment

PabloMansanet Apr 27, 2023

Choose a reason for hiding this comment

PabloMansanet left a comment

Choose a reason for hiding this comment

goodhoko commented Sep 15, 2024

goodhoko commented Apr 26, 2023 •

edited

Loading

strohel left a comment •

edited

Loading