Skip to content

Provide prebuilt native libraries by NuGet Packages.#257

Open
ha-ves wants to merge 10 commits into
NetCordDev:alphafrom
ha-ves:feature/prebuilt-natives
Open

Provide prebuilt native libraries by NuGet Packages.#257
ha-ves wants to merge 10 commits into
NetCordDev:alphafrom
ha-ves:feature/prebuilt-natives

Conversation

@ha-ves
Copy link
Copy Markdown

@ha-ves ha-ves commented Feb 9, 2026

Summary

Continues from #245. Implements CI-driven distribution of prebuilt native binary dependencies.

Build NetCord.Natives

  • Download size: ~112 MB
  • Extracted size: ~429 MB

Need to Follow-Up

  • DOCS: instructions for referencing prebuilt packages in projects
  • DOCS: fallback build guide for unsupported platforms or custom RIDs
  • FIX MACOS CI TEST:
dotnet test Tests/NetCord.Natives.Tests/ -tl:off -clp:NoSummary,Verbosity=normal -v normal -bl

and then use https://github.com/JanKrivanek/MSBuildStructuredLog

AllLibraryImportsExistInBinary tests are skipped on OSX due to ProcessModule limitation with dyld.

Included in This PR

✓ CI multi-platform native package build and publish
✓ Per-RID targeted package distribution
✓ Direct integration usage
✓ Custom vcpkg overlay ports for libdave and mlspp &
✓ Dependencies pinning
✓ Basic Unit & Integration tests

Overview

This PR provides prebuilt native libraries via NuGet as per-RID packages. Each package bundles:

  • Runtime binaries for standard .NET consumption
  • Static libraries for NativeAOT compilation
  • MSBuild integration to transparently resolve native paths based on target platform
  • Bundled licenses for all included dependencies

Packages are published automatically on successful CI builds to the configured NuGet feed.

Structure & Packaging Model

Per-RID Distribution Model:

  • Packages are produced per target RID (local or CI builds)
  • Package ID format: NetCord.Natives.{win|linux|osx}-{x64|arm64}
  • Example: NetCord.Natives.linux-x64, NetCord.Natives.win-arm64

Package Contents (per-RID package):

runtimes/{rid}/native/              ← Runtime binaries for standard .NET
staticlibs/{rid}/                   ← Static libraries for NativeAOT builds
build/NetCord.Natives.{rid}.*       ← MSBuild props/targets per platform
licenses/                           ← Copyright files from vcpkg ports

What Changed

CI Pruning:

Core Build & Packaging

  • NetCord.Natives.csproj: VCPKG build and use integration project
  • Custom vcpkg ports (natives-ports/):
    • libdave/portfile.cmake — discord/libdave with MSVC ARM64 patch
    • mlspp/portfile.cmake — cisco/mlspp, libdave's dependency

NuGet Package build files

Accessible Metadata

  • NativesHelper.cs: NativeLibraryVersionAttribute for runtime version discovery of bundled native libs

For issue tracking, refer to nativelibs versions in NuGet package description.

How to Use

Refer to docs included with this PR.

Maintainer Notes: vcpkg Baseline & Dependency Pinning

All native dependencies are pinned to a specific vcpkg baseline (vcpkg.json) to ensure reproducible builds.

Current constraint: openssl==3.0.7

When upgrading native library versions:

  1. Update version constraints in vcpkg.json
  2. Run local vcpkg build to validate against all RID triplets
  3. Verify if licenses are still viable

Resource requirements:

  • Build Runner Availability: On github hosted runner, first-time (cold cache) run can take up to ~30 min

Testing

Comprehensive native library validation via Tests/NetCord.Natives.Tests/:

Unit Tests (NativesBuildTests.cs)

Framework: MSTest with method-level parallelization
Target: .NET 10.0

NativeLoaded — Runtime Library Resolution

[TestMethod]
[DataRow("libdave")]
[DataRow("libsodium")]
[DataRow("opus")]
[DataRow("zstd")]
public void NativeLoaded(string libName)
  • Validates each of the four core native libraries can be successfully loaded via NativeLibrary.Load()
  • Catches load failures, missing binaries, or corrupted artifacts

AllLibraryImportsExistInBinary — P/Invoke Symbol Verification

Note: Skipped on OSX due to ProcessModule limitation with dyld preventing export enumeration. NativeAOT static linking tests still validate symbol presence on OSX.

[TestMethod]
[OSCondition(ConditionMode.Exclude, OperatingSystems.OSX)]
[DataRow("libdave", "NetCord.Gateway.Voice.Dave")]
[DataRow("libsodium", "NetCord.Gateway.Voice.Encryption.XChaCha20Poly1305")]
[DataRow("opus", "NetCord.Gateway.Voice.Opus")]
[DataRow("zstd", "NetCord.Gateway.Compression.Zstandard")]
public void AllLibraryImportsExistInBinary(string libName, string className)
  • Reflects over managed P/Invoke wrapper classes to extract all [LibraryImport] entry points
  • Loads each native binary and validates every exported symbol exists via NativeLibrary.TryGetExport()
  • Catches missing entry points or mismatched P/Invoke declarations

NativeAotStaticLinking — NativeAOT Full Integration

[TestMethod]
[DataRow("libdave;libsodium;opus;zstd")]
public void NativeAotStaticLinking(string libName)
  • Build NativeAot: Invokes dotnet publish -c Release on NativeAotApp with <PublishAot> enabled:
    • Passes current RID (e.g., win-x64, linux-x64, osx-arm64)
    • Links static libraries via <DirectPInvoke>
  • Runtime validation: Executes the published AOT app:
    • All native calls are valid
    • Exit code is 0
    • Logs build and run output for debugging

@ha-ves ha-ves force-pushed the feature/prebuilt-natives branch 2 times, most recently from 1c05c69 to 6565a35 Compare May 5, 2026 12:47
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 5, 2026

The documentation preview is available at https://preview.netcord.dev/257.

@AraHaan
Copy link
Copy Markdown

AraHaan commented May 6, 2026

I feel like the best option is to do something like:
NetCord.Natives.<rid> that way only the target RID's static libraries are downloaded and used for NAOT and does not slow down the build with extracting unused native static libraries.

ha-ves added 9 commits May 9, 2026 01:32
* put into nuget package
* Add native ports for libdave and mlspp
* Update build workflows for native packaging
* Add local targets and CI helpers for natives
* Update README
- GitHub Actions tag triggers, NuGet publish job
- NativesHelper, NativeLibraryVersionAttribute
- vcpkg integration, static linking
- zstd support
- native library tests, import checks
- Native AOT test app
- NetCordNativesDir assembly metadata
* Add MSVC builtin add overflow patch for libdave
* Add vcpkg-non-windows.targets for cross-platform support
* Update libdave and mlspp portfiles with build fixes
* Update build-natives workflow and vcpkg.json versions
* Fix vcpkg dependency resolution for multi-platform builds
* Add dynamic library exclusion for AOT builds
* Add NativeAotApp test project
* natives conditionals fix
* fix nativeaot test structure
* add natives tests to build workflow
- add reusable vcpkg setup
- pack per-RID native nupkgs
- streamline NativeAOT test logging
- tighten native linking targets
@ha-ves ha-ves force-pushed the feature/prebuilt-natives branch from 66a1e8d to ee00a64 Compare May 8, 2026 18:10
@ha-ves
Copy link
Copy Markdown
Author

ha-ves commented May 8, 2026

@AraHaan by default CI workflow will provide per-RID packages now.

But by .csproj design it will also work If anyone for some reason still needs to package all in one.

Copy link
Copy Markdown
Member

@KubaZ2 KubaZ2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't went through all the MSBuild stuff. I kind of hate that it takes so much and I am afraid it may require changes when new versions of binaries get released. I guess there is no better way to handle that? I think most binaries just require a single make or 2 cmake commands to build, but I think most of the MSBuild stuff is just for copying the binaries in the correct place and licensing?

Comment thread Tests/NetCord.Natives.Tests/NativesBuildTests.cs
Comment thread NetCord.Natives/natives-ports/mlspp/portfile.cmake
Comment thread NetCord.Natives/NativesHelper.cs Outdated
Comment thread NetCord.Natives.slnx
Comment thread Tests/NetCord.Natives.Tests/NetCord.Natives.Tests.csproj Outdated
Comment thread .github/workflows/build.yml Outdated
@ha-ves
Copy link
Copy Markdown
Author

ha-ves commented May 9, 2026

@KubaZ2
I haven't went through all the MSBuild stuff. I kind of hate that it takes so much and I am afraid it may require changes when new versions of binaries get released. I guess there is no better way to handle that? I think most binaries just require a single make or 2 cmake commands to build, but I think most of the MSBuild stuff is just for copying the binaries in the correct place and licensing?

Vcpkg is the least time-consuming dependency manager, it can build and put it all together in a convenient path.

The previous version of this PR was just using CMake, but it won't get the dependencies for your NativeAOT.
And it gets many times more complicated than this vcpkg version.

you can refer to this when there's new version:

When upgrading native library versions:

  1. Update version constraints in vcpkg.json (or natives-port// and target specific version and patch)
  2. Run local vcpkg build to validate against all RID triplets
  3. Verify if licenses are still viable

* Skip testing for OSX due to dyld limitation
* Clean up natives packaging
* Docs updates for prebuilt natives
@ha-ves ha-ves marked this pull request as ready for review May 10, 2026 21:37
@ha-ves
Copy link
Copy Markdown
Author

ha-ves commented May 10, 2026

@KubaZ2, there are some limitations from comments, but I think this PR is ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants