Re: Guile Hoot v0.2.0 released!

[Daniel Skinner]

> I don't think I quite understand what's going on. Are you passing over
WebAssembly.Instance objects as the event data to the AudioWorklet?

Not the instance, but rather WebAssembly.Module via compileStreaming as the
instance is not serializable.

> Could you just instantiate everything in the main "thread" and pass over
ready to use instances?

My understanding is no; modules on the other hand can be serialized and
passed to AudioWorklet via postMessage. Once there, they can be
instantiated in the worklet.

In my research, this is a pervasive issue for projects targeting wasm that
stems from limited api availability in AudioWorklet (e.g. no networking, no
fetch due to thread priority given, other little things). It also appears
the situation is made intentional and won't be changing.

see for example:
https://github.com/WebAudio/web-audio-api/issues/1439

https://github.com/rustwasm/wasm-pack/issues/689

or search "wasm audioworklet fetch"

The situation in hoot could be improved by accounting for this and basing
the Scheme.load* of reflect.js around modules too instead of only fetchable
paths. What I did was the smallest diff possible to confirm workable by
adjusting instantiate_streaming and adding a Scheme.set_module_map but I
dont necessarily recommend this.

iirc and specific to AudioWorklet, the situation could be further improved
by making the contents of reflect.js work with import, but (again, iirc)
the import statement for AudioWorklet only currently works in firefox and I
didn't get a chance to see if its planned for other browsers; in this way,
I wouldn't need to copy/paste the contents into realtime_worklet.js

> I also noticed you are doing things with WebGL. You beat me to it. ;)

it "just worked"! But I need to write up some helpers as I explore forms of
api design to do something more interesting.

> At some point, I will be porting Chickadee to the web using WebGL and
Web Audio so your exploration here will help make that happen.

I've only been writing scheme for like 5 weeks so it's my process for
learning. I read through Chickadee like 3 weeks back when researching
people's conventions and I appreciate you making that available.

What would be neat is to also get guile-fibers going (guile-fibers on
desktop platform is what got me to dive in to begin with).

On Mon, Dec 11, 2023, 9:49 AM Thompson, David <dthompson2@worcester.edu>
wrote:

> Hi Daniel,
>
> On Mon, Dec 11, 2023 at 12:53 AM Daniel Skinner <daniel@dasa.cc> wrote:
> >
> > I hit a snag trying to use in an AudioWorklet so was resolving that
> instead of making something more interesting. I worked it out which
> required a change to the reflect.js instatiate_streaming to allow
> referencing already-compiled modules keyed by given path as that's the only
> practical way it seems to provide to an AudioWorklet.
> >
> > Put here for reference (see everything prefixed realtime.*):
> https://gitlab.com/daniel126/guile-hoot-ffi-demo
> >
> > The realtime_worklet.js has a copy of reflect.js at top that includes
> the changes mentioned and also a TODO of a potential issue within same
> function.
>
> I don't think I quite understand what's going on. Are you passing over
> WebAssembly.Instance objects as the event data to the AudioWorklet?
> Could you just instantiate everything in the main "thread" and pass
> over ready to use instances?
>
> > Tested in firefox. The example uses AudioWorklet as a sink only; the raw
> audio data is generated in the guile wasm module by writing into a constant
> bytevector, then in AudioWorklet that data is copied-projected-copied out;
> maybe there's a cleaner way to do that copy out.
> >
> > With things spiked out now, I can maybe do something a little more
> interesting when I can find the time.
>
> I also noticed you are doing things with WebGL. You beat me to it. ;)
>
> At some point, I will be porting Chickadee to the web using WebGL and
> Web Audio so your exploration here will help make that happen.
>
> Thanks for sharing your progress!
>
> - Dave
>