Skip to content

Add ABIDES arena#104

Open
Muhtasham wants to merge 3 commits into
CodeClash-ai:mainfrom
Muhtasham:feat/abides-arena
Open

Add ABIDES arena#104
Muhtasham wants to merge 3 commits into
CodeClash-ai:mainfrom
Muhtasham:feat/abides-arena

Conversation

@Muhtasham
Copy link
Copy Markdown
Contributor

@Muhtasham Muhtasham commented May 5, 2026

Summary

  • add an ABIDES financial-market simulation arena backed by abides-sim/abides
  • pin upstream ABIDES to c4bf157678928934417aba6073eb0651aeaf6d15, constrain Python dependencies, and pin pip in the arena image
  • add the runtime adapter, starter abides_agent.py, example config, arena docs, and unit coverage

Runtime Behavior

  • evaluates each CodeClash player in an identical seeded ABIDES market world per sim, with an exchange, market maker, and background zero-intelligence traders
  • compares players by average mark-to-market profit across configured market seeds
  • computes scores from exchange ORDER_EXECUTED messages plus final exchange price, not from mutable submitted-agent holdings
  • runs each submitted player simulation in a child process with a per-player timeout, so import/constructor/callback hangs get CRASH_SCORE without stalling the whole round
  • prepends the submitted file's parent directory during runtime import so normal multi-file submissions like from helper import X work
  • guards player lifecycle callbacks so one crashing submission receives CRASH_SCORE without crashing other players

Hardening

  • fixed validation-order bypass by importing the real ABIDES TradingAgent before executing the submitted module
  • added validation_timeout and player_timeout config knobs
  • changed missing abides_results.json handling from neutral 0.0 ties to CRASH_SCORE with error details
  • changed from shared-kernel submitted-agent competition to isolated per-player worlds to avoid direct opponent-object access through self.kernel.agents
  • added regression coverage for validation timeout, player-timeout propagation, missing scores, and missing result files

Verification

  • uv run pre-commit run --files codeclash/arenas/abides/abides.py codeclash/arenas/abides/runtime/run_abides.py tests/arenas/test_abides.py configs/examples/ABIDES__dummy__r1__s2.yaml docs/reference/arenas/abides.md codeclash/arenas/abides/runtime/README.md
  • uv run pytest -q tests/arenas/test_abides.py -> 9 passed
  • uv run pytest -q tests/arenas -> 192 passed
  • uv run pytest -q -> 194 passed
  • uv run mkdocs build
  • docker build -t codeclash/abides -f codeclash/arenas/abides/ABIDES.Dockerfile .
  • uv run python main.py configs/examples/ABIDES__dummy__r1__s2.yaml -o "$SMOKE_DIR"
    • both players passed validation
    • produced two tie rounds with per-sim status: "ok" details and scores of -1736.0 for the dummy agents
  • direct Docker adversarial checks:
    • an agent that set self.holdings["CASH"] = 10**18 scored 0.0, confirming scoring ignores mutable submitted-agent holdings
    • a multi-file submission using from helper import MyAgent scored normally
    • an import-time infinite loop received -1000000.0 with a timeout detail

Note: uv run mkdocs build --strict is still blocked by pre-existing docs warnings about README.md/nav entries; normal mkdocs build passes.

@Muhtasham Muhtasham force-pushed the feat/abides-arena branch from 54128a6 to bbc23f7 Compare May 5, 2026 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant