Skip to content

[INFRAHELP-2899] fix: address race condition in load_nested_zip() and add tests#15

Merged
ajaxbits merged 5 commits into
mainfrom
INFRAHELP-2899-fix-race
May 7, 2026
Merged

[INFRAHELP-2899] fix: address race condition in load_nested_zip() and add tests#15
ajaxbits merged 5 commits into
mainfrom
INFRAHELP-2899-fix-race

Conversation

@ajaxbits
Copy link
Copy Markdown
Collaborator

@ajaxbits ajaxbits commented May 7, 2026

Fixes a race condition in nested_zip_loader where, if multiple processes are trying to run load_nested_zip() a race condition could occur. Two processes would try to call os.rename at the same time, which would fail if the target_package_path was not empty.

To get around this, we create a filesystem lockfile. Each time load_nested_zip() runs, it has to acquire the lock, ensuring that concurrent unzips are impossible.

A basic set of tests are also added to assert this behavior. It tests that cold starts always extract, warm starts skip the extraction step, nothing is left behind, and that multiple processes running at the same time don't create a race condition.

Note

We have to use importlib in the tests because nested_zip_loader.py calls load_nested_zip() at import time. This is essential to its function and can't be worked around. Therefore, we have to do some shenanigans with importlib to be able to actually test load_nested_zip() in isolation on test data.

Finally, we fix a small bug where we never explicitly closed zipfile.ZipFile.

@ajaxbits ajaxbits requested a review from Copilot May 7, 2026 19:10
@ajaxbits ajaxbits changed the title [INFRAHELP-2899] test: add unit tests for nested_zip_loader [INFRAHELP-2899] fix: address race condition in load_nested_zip() and add tests May 7, 2026
@ajaxbits ajaxbits force-pushed the INFRAHELP-2899-fix-race branch 2 times, most recently from 84fd64b to 4d3a673 Compare May 7, 2026 19:13
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a reported race condition in package_python_function/nested_zip_loader.py by serializing extraction/rename with a filesystem lock, and introduces a focused test suite to validate cold-start/warm-start behavior and concurrent imports.

Changes:

  • Add an fcntl.flock-based lockfile around nested dependency extraction/rename to prevent concurrent processes from racing.
  • Add tests/test_nested_zip_loader.py to validate extraction behavior, staging cleanup, and a basic multi-process concurrency scenario.
  • Remove the prior packaging/CLI test suite (tests/test_package_python_function.py, tests/conftest.py, tests/test_local.py) and add .opencode to .gitignore.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
package_python_function/nested_zip_loader.py Adds a lockfile to avoid concurrent extraction/rename races; minor formatting changes.
tests/test_nested_zip_loader.py New unit tests for loader cold/warm starts, staging cleanup, and concurrency.
tests/test_package_python_function.py Removed (previous end-to-end packaging/reproducibility tests).
tests/conftest.py Removed (previous shared test data/fixtures for packaging tests).
tests/test_local.py Removed (previous local/manual test helper).
.gitignore Ignores .opencode.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/test_nested_zip_loader.py
Comment thread tests/test_nested_zip_loader.py
Comment thread package_python_function/nested_zip_loader.py Outdated
Comment thread package_python_function/nested_zip_loader.py Outdated
@ajaxbits ajaxbits force-pushed the INFRAHELP-2899-fix-race branch 3 times, most recently from 332f77b to afe566d Compare May 7, 2026 19:29
@ajaxbits ajaxbits force-pushed the INFRAHELP-2899-fix-race branch 2 times, most recently from ede27cc to 5b1a7f9 Compare May 7, 2026 19:31
@ajaxbits ajaxbits force-pushed the INFRAHELP-2899-fix-race branch from 5b1a7f9 to 1300f00 Compare May 7, 2026 19:32
@ajaxbits ajaxbits requested a review from Copilot May 7, 2026 19:32
@ajaxbits ajaxbits requested a review from BrandonLWhite May 7, 2026 19:34
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.

Comment thread tests/test_nested_zip_loader.py
Comment thread tests/test_nested_zip_loader.py Outdated
Comment thread package_python_function/nested_zip_loader.py Outdated
ajaxbits and others added 2 commits May 7, 2026 14:37
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@ajaxbits ajaxbits marked this pull request as ready for review May 7, 2026 19:37
Copy link
Copy Markdown
Owner

@BrandonLWhite BrandonLWhite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a very good solution to the problem. Thanks!

@ajaxbits ajaxbits merged commit da69112 into main May 7, 2026
1 check passed
@ajaxbits ajaxbits deleted the INFRAHELP-2899-fix-race branch May 7, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants