feat(rerank): per-pool rerank callback for vector search#57
Merged
Conversation
Adds opt-in RerankConfig { multiplier, callback } on EmbeddingConfig.
Pipeline fetches Math.floor(limit * multiplier) candidates, hands them
to the user callback, then trims to limit. Errors propagate.
10-task TDD plan covering types, wiring, validation, tests, README, and changeset. Implements the design at docs/superpowers/specs/2026-05-18-rerank-callback-design.md.
- Task 1: move RerankFn/RerankConfig re-export here so Task 2's red test fails on the wiring assertion, not module resolution - Task 2: baseline call now uses a no-rerank handlers instance so the callback fires exactly once and the reversal assertion is meaningful - Task 3: non-rerank path forwards `limit` as-is (no `?? 10`), preserving existing behavior for direct payload-API callers; `?? 10` only applies on the rerank path
….callback - Merge orphan tail-note into bulkEmbeddingsFns JSDoc where it semantically belongs (also fixes a stale name reference) - Add JSDoc to RerankConfig.callback for consistency with other *Fn fields
…pulated Without cron+autoRun, vectorization jobs queue but never execute, leaving the mock adapter empty. Test 1's ordering assertion then compares [].reverse() to [] — trivially equal. Mirror the autoRun pattern used in vectorSearch.spec.ts so search() returns real seeded chunks.
- Defend the example callback against undefined from results[r.index] - Split trailing paragraph into Multiplier / Limit / Errors / Validation and call out the default-limit divergence vs the non-rerank path - Add Reranking entry to the Features list and Table of Contents
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rerank: { multiplier, callback }on each pool'sembeddingConfig. When set, the plugin fetchesMath.floor(limit * multiplier)candidates from the adapter, hands them to the user-supplied callback, and trims the callback's output to the caller'slimit. Provider-agnostic — bring your own Voyage / Cohere / local cross-encoder.limitis forwarded to the adapter as-is (possiblyundefined), preserving existing direct payload-API caller behavior. The?? 10default only applies inside the rerank branch.multipliermust be a finite number>= 1(floats allowed,Math.floorapplied). Invalid configs are rejected at plugin init with a pool-named error. Callback errors propagate to the caller — no silent fallback.Test plan
pnpm test:int dev/specs/vectorSearchRerank.spec.ts— 8 tests: callback invocation + ordering, multiplier (int and float) expanding fetch,limitomitted defaults to 10, no-rerank baseline, trim when callback returns more than limit, return-fewer when callback returns fewer, error propagation.pnpm test:int dev/specs/rerankValidation.spec.ts— 6 tests: multiplier 0 / -1 / NaN / Infinity / non-function callback all throw; valid config builds cleanly.pnpm test:int— full integration suite, 29 files / 81 tests passing.pnpm build:types— typecheck clean.