Skip to content

Remove Unity resampling by configuring native audio source correctly#271

Draft
MaxHeimbrock wants to merge 2 commits intomainfrom
max/audiostream-resample-rust
Draft

Remove Unity resampling by configuring native audio source correctly#271
MaxHeimbrock wants to merge 2 commits intomainfrom
max/audiostream-resample-rust

Conversation

@MaxHeimbrock
Copy link
Copy Markdown
Contributor

@MaxHeimbrock MaxHeimbrock commented May 5, 2026

Summary

Remove second resampling in Unity, by configuring the Rust audio stream with correct (channels, sampleRate) to begin with. The problem is, that in Unity you don't know the channel number of each audio source until it runs, so we need to delay the initialization of the native audio stream until audio is read in Unity.

Changes

  • Drop C# RemixAndResample per-frame path. OnAudioStreamEvent now just memcpys the inbound AudioFrame into the ring buffer — no FFI roundtrip on the OS-audio-scheduled callback thread. Rust's NativeAudioStream does the rate/channel conversion internally, once, with its existing pipeline.
  • Lazily create the FFI native stream from the first OnAudioRead. AudioSettings.GetConfiguration().speakerMode disagrees with what OnAudioFilterRead actually delivers when the AudioSource is routed differently (per-source mono routing, spatializers, mixer chains, headphone profiles) — observed on macOS Editor + Sony headphones (system Stereo, source mono → low-pitched playback). The first callback is the authoritative source for (channels, sampleRate); we post a one-shot CreateFfiStream(...) to the main thread via FfiClient._context, build the NewAudioStreamRequest with the observed values, and only then subscribe to AudioStreamEventReceived.
  • No reconfigure / handle swap path. Because we never create the stream with a wrong guess, there's no old/new stream window with in-flight frames to reconcile.

Files

  • Runtime/Scripts/AudioStream.cs_resampler field gone; Handle is now _handle behind a property; CreateFfiStream added; constructor only sets up AudioSource/AudioProbe/pause hook.
  • Tests/EditMode/MediaStreamLifetimeTests.cs — assertions updated for the renamed/removed members.

Profiler

Before change, after 5 minutes and multiple audio sources: Total Audio CPU reaches ~100%
Screenshot 2026-05-06 at 14 47 09


With change, after 5 minutes and multiple audio sources: Total Audio CPU stays below ~10%
Screenshot 2026-05-06 at 14 46 48

Comment thread Runtime/Scripts/AudioStream.cs Outdated
@MaxHeimbrock MaxHeimbrock force-pushed the max/audiostream-resample-rust branch from 90d0b04 to 3b570fc Compare May 6, 2026 06:38
@MaxHeimbrock MaxHeimbrock changed the title Max/audiostream resample rust Remove Unity resampling by configuring native audio source correctly May 6, 2026
MaxHeimbrock and others added 2 commits May 6, 2026 14:25
Configuring the native stream from AudioSettings.speakerMode at construction
time can disagree with what OnAudioFilterRead actually delivers (e.g. system
Stereo + per-source mono routing → Rust pushes 2ch but Unity wants 1ch, audio
plays low-pitched). Defer the NewAudioStreamRequest until the first audio
callback so we configure with Unity's authoritative (channels, sampleRate)
and never need a runtime reconfigure swap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@MaxHeimbrock MaxHeimbrock force-pushed the max/audiostream-resample-rust branch from 3b570fc to 337c769 Compare May 6, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant