Skip to content

Add verifiers integration bridge#680

Open
kkudumu wants to merge 2 commits into
OpenPipe:mainfrom
kkudumu:codex/verifiers-bridge
Open

Add verifiers integration bridge#680
kkudumu wants to merge 2 commits into
OpenPipe:mainfrom
kkudumu:codex/verifiers-bridge

Conversation

@kkudumu
Copy link
Copy Markdown

@kkudumu kkudumu commented May 14, 2026

Summary

Adds an optional art.verifiers bridge for Prime Intellect verifiers environments:

  • run a verifiers.Environment with an ART model and return an art.Trajectory
  • run grouped verifiers rollouts and return an art.TrajectoryGroup
  • convert serialized verifiers rollout outputs to ART trajectories
  • convert ART trajectories/groups back into verifiers-compatible rollout output dicts
  • document the integration under docs/integrations/verifiers-integration.mdx

This is aimed at the ART <-> verifiers portability path from the Algora ART-E / ART bounty without adding verifiers as a hard dependency for normal ART users.

Tests

  • python3 -m py_compile src/art/verifiers.py src/art/test/test_verifiers_bridge.py
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src pytest -q src/art/test/test_verifiers_bridge.py
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src python3 -m ruff check src/art/verifiers.py src/art/test/test_verifiers_bridge.py

@kkudumu
Copy link
Copy Markdown
Author

kkudumu commented May 14, 2026

@algora-pbc This PR targets the Prime Intellect ART-E / ART bounty, specifically the hard-path ART <-> verifiers task portability scope: https://algora.io/PrimeIntellect-ai/bounties/cxptwFoEv2up9Qjm

@kkudumu kkudumu force-pushed the codex/verifiers-bridge branch from 817c8d8 to d500de2 Compare May 14, 2026 08:50
@kkudumu
Copy link
Copy Markdown
Author

kkudumu commented May 14, 2026

Follow-up pushed:

  • expanded tests from 4 to 7, including async wrapper coverage for running single and grouped verifiers environment rollouts with an ART model client
  • kept the runtime import optional so base ART installs are unaffected

Verification after the update:

  • python3 -m py_compile src/art/verifiers.py src/art/test/test_verifiers_bridge.py
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src pytest -q src/art/test/test_verifiers_bridge.py -> 7 passed
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src python3 -m ruff check src/art/verifiers.py src/art/test/test_verifiers_bridge.py -> passed

@kkudumu kkudumu force-pushed the codex/verifiers-bridge branch from d500de2 to 214cc21 Compare May 14, 2026 08:56
@kkudumu
Copy link
Copy Markdown
Author

kkudumu commented May 14, 2026

Small CI-risk cleanup pushed after the test expansion:

  • removed the proposed pyproject.toml optional extra because this repo uses a frozen uv.lock; refreshing the lock in this environment is currently blocked by an existing Megatron build-metadata issue unrelated to this PR
  • kept verifiers as an optional runtime import and documented installation as uv pip install -U openpipe-art verifiers
  • retained the expanded async wrapper tests

Verification on the updated head:

  • python3 -m py_compile src/art/verifiers.py src/art/test/test_verifiers_bridge.py
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src pytest -q src/art/test/test_verifiers_bridge.py -> 7 passed
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src python3 -m ruff check src/art/verifiers.py src/art/test/test_verifiers_bridge.py -> passed
  • git diff --check HEAD -> passed

@kkudumu
Copy link
Copy Markdown
Author

kkudumu commented May 14, 2026

@dhruvrnaik quick visibility note for the Algora ART-E / ART bounty: this PR targets the hard-scope ART <-> verifiers portability path, not just the local ART-E example path.

Current verification on this PR:

  • python3 -m py_compile src/art/verifiers.py src/art/test/test_verifiers_bridge.py
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src pytest -q src/art/test/test_verifiers_bridge.py -> 7 passed
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src python3 -m ruff check src/art/verifiers.py src/art/test/test_verifiers_bridge.py -> passed
  • git diff --check HEAD -> passed

The integration keeps verifiers optional, so normal ART installs are unaffected while users who install verifiers get the bridge APIs.

@kkudumu
Copy link
Copy Markdown
Author

kkudumu commented May 14, 2026

Follow-up pushed in 28e59c7 to make the hard-scope 2-way portability contract more explicit.

What changed:

  • Added normalize_verifiers_rollout_output() and normalize_verifiers_rollout_outputs().
  • These round-trip saved verifiers outputs through ART trajectories and back into verifiers-compatible output shape.
  • The normalizer preserves prompt/completion message shape, reward, numeric metrics, tool definitions, logs, completion/truncation state, timing, answer, and stop-condition metadata while carrying ART metadata in the optional trajectory column.
  • Added round-trip tests and docs showing the portability check.

Verification rerun:

  • python3 -m py_compile src/art/verifiers.py src/art/test/test_verifiers_bridge.py
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src pytest -q src/art/test/test_verifiers_bridge.py -> 9 passed
  • PYTHONPATH=/tmp/art-verifiers-test-deps:src python3 -m ruff check src/art/verifiers.py src/art/test/test_verifiers_bridge.py -> passed
  • git diff --check -> passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant