feat(cluster): add cluster-wide key browser (GET /v1/cache/keys)#134
Merged
Conversation
Introduce a new operator-debug endpoint that fans out across every alive peer, deduplicates replicas, sorts, and returns a paged key set. Pattern matching supports two modes via a classifier in buildKeyMatcher: patterns without glob metacharacters use strings.HasPrefix (prefix mode); patterns containing `*`, `?`, or `[` use path.Match (glob mode). Invalid globs are rejected at construction and surface as 400 BAD_REQUEST. Hard caps bound worst-case memory and response size: - `max` (default 10000, ceiling 50000): deduplicated result set cap - `limit` (default 100, ceiling 500): page size; cursor is offset-based Per-peer fan-out failures are best-effort — failed peer IDs land in `partial_nodes`, consistent with read-repair/hint-replay contracts. Returns 501 when the backend is not DistMemory. Route registered before /v1/cache/:key to prevent Fiber's trie router from shadowing it. Core changes: - pkg/backend/dist_keys.go: ListKeys fan-out via listKeysAccumulator (mutex-guarded dedup map) + localMatchingKeys for self-peer shard scan - DistTransport interface extended: ListKeys(ctx, nodeID, pattern string) - InProcessTransport, DistHTTPTransport, and chaosTransport implementations - /internal/keys extended with optional `q` param (backward compatible) - collectShardKeys accepts a matcher; non-matching keys skip the limit - HyperCache.ClusterKeys added as the public entry point Tests: 12-case unit table for buildKeyMatcher, HTTP smoke tests for paged walk and 400 surfaces, five integration tests covering cluster-wide dedup at RF=3 (50 seeds → 50 keys, not 150), prefix/glob filters, and max-cap truncation. OpenAPI spec and drift-detector updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a new operator-debug endpoint that fans out across every alive peer, deduplicates replicas, sorts, and returns a paged key set.
Pattern matching supports two modes via a classifier in buildKeyMatcher: patterns without glob metacharacters use strings.HasPrefix (prefix mode); patterns containing
*,?, or[use path.Match (glob mode). Invalid globs are rejected at construction and surface as 400 BAD_REQUEST.Hard caps bound worst-case memory and response size:
max(default 10000, ceiling 50000): deduplicated result set caplimit(default 100, ceiling 500): page size; cursor is offset-basedPer-peer fan-out failures are best-effort — failed peer IDs land in
partial_nodes, consistent with read-repair/hint-replay contracts. Returns 501 when the backend is not DistMemory. Route registered before /v1/cache/:key to prevent Fiber's trie router from shadowing it.Core changes:
qparam (backward compatible)Tests: 12-case unit table for buildKeyMatcher, HTTP smoke tests for paged walk and 400 surfaces, five integration tests covering cluster-wide dedup at RF=3 (50 seeds → 50 keys, not 150), prefix/glob filters, and max-cap truncation. OpenAPI spec and drift-detector updated.