Skip to content

fix(s3): paginate ListObjectsV2 for getBundleHistory (s3.ts)#164

Open
FastHaleqt wants to merge 1 commit intobase:masterfrom
FastHaleqt:fix/s3
Open

fix(s3): paginate ListObjectsV2 for getBundleHistory (s3.ts)#164
FastHaleqt wants to merge 1 commit intobase:masterfrom
FastHaleqt:fix/s3

Conversation

@FastHaleqt
Copy link
Copy Markdown

Problem

getBundleHistory lists every object under the S3 prefix bundles/{bundleKey}/ and then loads each object to build the bundle’s event history. It does this with Amazon S3 ListObjectsV2, which returns at most ~1000 keys per request. If that prefix has more than one page of keys, the old code only read the first page. Any events whose keys appeared on later pages were never listed and never loaded, so the UI/API showed an incomplete bundle history for large bundles.

Why this matters

Bundle history is used to understand what happened to a user’s transaction (e.g. metering, lifecycle events). Silently dropping events after the first 1000 keys is hard to notice in normal testing but wrong in production for high-volume or long-running bundles. Pagination is the standard S3 pattern to list an unbounded number of objects under a prefix.

What we changed

  • listAllObjectKeysForPrefix(s3, bucket, prefix) (in src/lib/s3.ts, exported) — follows IsTruncated / NextContinuationToken and passes ContinuationToken on the next call until all keys under the prefix are collected.
  • getBundleHistory now uses that helper for bundles/{bundleKey}/, then unchanged per-key GetObject + JSON parsing. Semantics are the same as before; we only fix completeness of the key list.
  • Bucket and client are unchanged (TIPS_UI_S3_BUCKET_NAME / default tips, same s3Client setup as the rest of the file).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant