Entropy

[pelikhan]

/scout investigate how to leverage
https://github.com/juyterman1000/entroly in a agentic workflow

[github-actions]

🔭 Recon complete! Scout has charted the territory. Map ready! 🗺️

[github-actions]

🔍 Scout Research Report

Triggered by @pelikhan

Executive Summary

Entroly is a codebase context compression library (Python + Rust/WASM) designed to reduce AI token costs by 70–95%. It indexes an entire repo, ranks files by information density, and delivers a compressed, budget-aware context window to any LLM. For agentic workflows, it offers five integration surfaces: an MCP server, a Python SDK, an HTTP proxy, a LangChain LCEL runnable, and a higher-level FlowOrchestrator with five built-in epistemic flows.

Click to expand detailed findings

Research Findings

What Entroly Does

Entroly solves the "5% visibility" problem: most AI coding tools only see 5–10 files per request. Entroly maps and compresses your entire codebase into the agent's context window at variable resolution:

• Full resolution → critical files
• Signature-level → supporting files
• Reference-only → everything else

The compression engine is written in Rust and compiled to WASM (entroly-core), with a Python wrapper. Claims <10 ms latency per call.

---

Integration Pattern 1 — MCP Server (Claude Code, Cursor, Copilot)

# Claude Code
claude mcp add entroly -- entroly

# Any MCP-native tool
entroly init  # auto-detects and writes MCP server config

Once registered, the running MCP server serves compressed codebase context to any MCP-compatible agent without any code changes.

---

Integration Pattern 2 — Python SDK (drop-in for any agent)

from entroly import compress, compress_messages

# Compress a single chunk (e.g. a RAG retrieval result)
compressed = compress(big_context, budget=2000)

# Compress an entire conversation history
messages = compress_messages(messages, budget=50_000, preserve_last_n=4)

compress_messages is particularly useful in long-running agentic loops where conversation history grows unbounded — it compresses older turns aggressively while preserving the last N messages verbatim and always keeping the last user message intact.

---

Integration Pattern 3 — LangChain / LCEL Middleware

from langchain_openai import ChatOpenAI
from entroly.integrations.langchain import EntrolyCompressor

llm = ChatOpenAI(model="claude-opus-4-7")  # or any model
chain = EntrolyCompressor(budget=30_000) | llm

# Works with full LCEL chains too
chain = prompt | EntrolyCompressor(budget=30_000) | llm | output_parser

EntrolyCompressor is a Runnable-compatible middleware. It handles str, list[dict], and LangChain BaseMessage lists transparently.

---

Integration Pattern 4 — HTTP Proxy (language-agnostic)

entroly proxy  # starts proxy on localhost:9377

Point any LLM client at `(localhost/redacted) instead of the real API endpoint. Entroly intercepts requests, compresses the context, and forwards to the real endpoint. Zero code changes in the agent itself.

---

Integration Pattern 5 — Flow Orchestrator (structured agentic reasoning)

The FlowOrchestrator (in entroly/flow_orchestrator.py) implements five canonical epistemic flows, making it a lightweight agentic framework in its own right:

Flow	Steps	When to use
Fast Answer	Belief → Action	High-confidence, latency-sensitive
Verify Before Answer	Belief → Verification → Action	When accuracy matters
Compile On Demand	Truth → Belief → Verification → Action	First-run or stale cache
Change-Driven	Event → Truth → Belief → Verification → Action	Git push / PR triggers
Self-Improvement	Misses → Verification → Evolution → Belief	When the agent is getting things wrong

from entroly.flow_orchestrator import FlowOrchestrator
from entroly.epistemic_router import EpistemicRouter

router = EpistemicRouter(vault)
orchestrator = FlowOrchestrator(vault, router, compiler, verifier, ...)
result = orchestrator.execute("Why is auth broken?")
print(result.answer)

The orchestrator automatically routes to the right flow based on query type, then closes a feedback loop by recording whether the flow succeeded — enabling adaptive threshold tuning over time.

---

CI/CD Integration

# .github/workflows/ai-cost-check.yml
- uses: juyterman1000/entroly-cost-check-`@v1`

The companion GitHub Action (entroly-cost-check) checks token regressions in every PR.

---

Self-Evolving + Federated Learning

Entroly maintains a vault (.entroly/vault/) of learned "beliefs" — structured facts about your codebase compiled from source. A background daemon (evolution_daemon.py) benchmarks compression strategies and promotes the best-performing ones. The learning budget is capped at 5% × lifetime_savings — it cannot spend more tokens on learning than it saves.

An opt-in federation layer (ENTROLY_FEDERATION=1) shares optimization weights (not code) across instances via GitHub. Only anonymous weight deltas leave the machine.

---

Limitations and Caveats

• Repository is relatively young (no obvious stable release yet — check PyPI pip install entroly for current version).
• The Rust core (entroly-core) has a clippy_errors.txt checked in, suggesting the native engine may not always build cleanly; the Python fallback path is available.
• The federated learning feature is opt-in but worth understanding before enabling in regulated environments.
• Benchmark data (95%+ token savings, 484 tests) is self-reported in the README — independent validation is not available yet.

---

Recommendations

1. Start with the MCP server if you are using Claude Code or Cursor — it requires zero code changes and gives immediate gains: claude mcp add entroly -- entroly.

2. Use compress_messages in any long-running agentic loop to prevent context overflow without losing conversation coherence.

3. Use EntrolyCompressor | llm as a drop-in optimization layer in LangChain or similar LCEL-style pipelines.

4. Consider the FlowOrchestrator if you want structured epistemic flows with built-in self-improvement — particularly the change_driven flow for event-triggered agents (e.g., on PR creation).

5. Add the GitHub Action (entroly-cost-check-@v1``) to your CI pipeline to catch token-cost regressions early.

Key Sources

• Entroly GitHub Repository
• Entroly SDK (sdk.py)
• LangChain Integration
• Flow Orchestrator
• entroly-cost-check GitHub Action

Suggested Next Steps

1. pip install entroly and run entroly go in this repo to see a live compression report.
2. Add claude mcp add entroly -- entroly to your Claude Code setup for zero-config context compression.
3. Evaluate compress_messages in any agentic loops that call the Anthropic API repeatedly.
4. Review the FlowOrchestrator flows to see if the change-driven or self-improvement flows fit your automation needs.

References:

• §24841892836

🔭 Intelligence gathered by Scout · ● 148.2K · ◷