Skip to content

feat(cluster): add cluster-wide key browser (GET /v1/cache/keys)#134

Merged
hyp3rd merged 1 commit into
mainfrom
feat/dist-cluster
May 13, 2026
Merged

feat(cluster): add cluster-wide key browser (GET /v1/cache/keys)#134
hyp3rd merged 1 commit into
mainfrom
feat/dist-cluster

Conversation

@hyp3rd
Copy link
Copy Markdown
Owner

@hyp3rd hyp3rd commented May 13, 2026

Introduce a new operator-debug endpoint that fans out across every alive peer, deduplicates replicas, sorts, and returns a paged key set.

Pattern matching supports two modes via a classifier in buildKeyMatcher: patterns without glob metacharacters use strings.HasPrefix (prefix mode); patterns containing *, ?, or [ use path.Match (glob mode). Invalid globs are rejected at construction and surface as 400 BAD_REQUEST.

Hard caps bound worst-case memory and response size:

  • max (default 10000, ceiling 50000): deduplicated result set cap
  • limit (default 100, ceiling 500): page size; cursor is offset-based

Per-peer fan-out failures are best-effort — failed peer IDs land in partial_nodes, consistent with read-repair/hint-replay contracts. Returns 501 when the backend is not DistMemory. Route registered before /v1/cache/:key to prevent Fiber's trie router from shadowing it.

Core changes:

  • pkg/backend/dist_keys.go: ListKeys fan-out via listKeysAccumulator (mutex-guarded dedup map) + localMatchingKeys for self-peer shard scan
  • DistTransport interface extended: ListKeys(ctx, nodeID, pattern string)
  • InProcessTransport, DistHTTPTransport, and chaosTransport implementations
  • /internal/keys extended with optional q param (backward compatible)
  • collectShardKeys accepts a matcher; non-matching keys skip the limit
  • HyperCache.ClusterKeys added as the public entry point

Tests: 12-case unit table for buildKeyMatcher, HTTP smoke tests for paged walk and 400 surfaces, five integration tests covering cluster-wide dedup at RF=3 (50 seeds → 50 keys, not 150), prefix/glob filters, and max-cap truncation. OpenAPI spec and drift-detector updated.

Introduce a new operator-debug endpoint that fans out across every alive
peer, deduplicates replicas, sorts, and returns a paged key set.

Pattern matching supports two modes via a classifier in buildKeyMatcher:
patterns without glob metacharacters use strings.HasPrefix (prefix mode);
patterns containing `*`, `?`, or `[` use path.Match (glob mode). Invalid
globs are rejected at construction and surface as 400 BAD_REQUEST.

Hard caps bound worst-case memory and response size:
- `max` (default 10000, ceiling 50000): deduplicated result set cap
- `limit` (default 100, ceiling 500): page size; cursor is offset-based

Per-peer fan-out failures are best-effort — failed peer IDs land in
`partial_nodes`, consistent with read-repair/hint-replay contracts.
Returns 501 when the backend is not DistMemory. Route registered before
/v1/cache/:key to prevent Fiber's trie router from shadowing it.

Core changes:
- pkg/backend/dist_keys.go: ListKeys fan-out via listKeysAccumulator
  (mutex-guarded dedup map) + localMatchingKeys for self-peer shard scan
- DistTransport interface extended: ListKeys(ctx, nodeID, pattern string)
- InProcessTransport, DistHTTPTransport, and chaosTransport implementations
- /internal/keys extended with optional `q` param (backward compatible)
- collectShardKeys accepts a matcher; non-matching keys skip the limit
- HyperCache.ClusterKeys added as the public entry point

Tests: 12-case unit table for buildKeyMatcher, HTTP smoke tests for paged
walk and 400 surfaces, five integration tests covering cluster-wide dedup
at RF=3 (50 seeds → 50 keys, not 150), prefix/glob filters, and max-cap
truncation. OpenAPI spec and drift-detector updated.
@hyp3rd hyp3rd merged commit 8f87516 into main May 13, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant