Remove Unity resampling by configuring native audio source correctly#271
Draft
MaxHeimbrock wants to merge 2 commits intomainfrom
Draft
Remove Unity resampling by configuring native audio source correctly#271MaxHeimbrock wants to merge 2 commits intomainfrom
MaxHeimbrock wants to merge 2 commits intomainfrom
Conversation
MaxHeimbrock
commented
May 6, 2026
90d0b04 to
3b570fc
Compare
Configuring the native stream from AudioSettings.speakerMode at construction time can disagree with what OnAudioFilterRead actually delivers (e.g. system Stereo + per-source mono routing → Rust pushes 2ch but Unity wants 1ch, audio plays low-pitched). Defer the NewAudioStreamRequest until the first audio callback so we configure with Unity's authoritative (channels, sampleRate) and never need a runtime reconfigure swap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3b570fc to
337c769
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Remove second resampling in Unity, by configuring the Rust audio stream with correct
(channels, sampleRate)to begin with. The problem is, that in Unity you don't know the channel number of each audio source until it runs, so we need to delay the initialization of the native audio stream until audio is read in Unity.Changes
RemixAndResampleper-frame path.OnAudioStreamEventnow just memcpys the inboundAudioFrameinto the ring buffer — no FFI roundtrip on the OS-audio-scheduled callback thread. Rust'sNativeAudioStreamdoes the rate/channel conversion internally, once, with its existing pipeline.OnAudioRead.AudioSettings.GetConfiguration().speakerModedisagrees with whatOnAudioFilterReadactually delivers when the AudioSource is routed differently (per-source mono routing, spatializers, mixer chains, headphone profiles) — observed on macOS Editor + Sony headphones (system Stereo, source mono → low-pitched playback). The first callback is the authoritative source for(channels, sampleRate); we post a one-shotCreateFfiStream(...)to the main thread viaFfiClient._context, build theNewAudioStreamRequestwith the observed values, and only then subscribe toAudioStreamEventReceived.Files
Runtime/Scripts/AudioStream.cs—_resamplerfield gone;Handleis now_handlebehind a property;CreateFfiStreamadded; constructor only sets up AudioSource/AudioProbe/pause hook.Tests/EditMode/MediaStreamLifetimeTests.cs— assertions updated for the renamed/removed members.Profiler
Before change, after 5 minutes and multiple audio sources: Total Audio CPU reaches ~100%

With change, after 5 minutes and multiple audio sources: Total Audio CPU stays below ~10%
