Bug Description
`agentready submit` commits the assessment JSON verbatim, including fields that contain local machine information:
- `metadata.executed_by` — username and hostname (e.g. `user@hostname`)
- `metadata.working_directory` — absolute local path (e.g. `/home/user/git/org/repo`)
- `metadata.command` — may contain absolute paths
- `repository.path` — absolute local path
This means every leaderboard submission leaks the submitter's username, hostname, and local directory layout. The submitter in PR #424 caught this only because CodeRabbit flagged it and manually redacted the fields in a follow-up commit.
Older submissions in `submissions/` still contain unredacted PII.
Root Cause
In `src/agentready/cli/submit.py`, both submission paths (`submit_with_gh_cli` lines 352-356 and `submit_with_token` lines 564-565) read the assessment file and commit it as-is:
```python
with open(assessment_path, encoding="utf-8") as f:
content = f.read()
```
There is no sanitization step.
Expected Behavior
`agentready submit` automatically strips local PII before committing the file to the leaderboard. Users should not need to manually review or redact their assessment before submitting.
Possible Solution
Add a `sanitize_for_submission(data: dict) -> dict` function in `submit.py` called right after `load_assessment()`. It should replace:
| Field |
Replace with |
| `metadata.executed_by` |
`"redacted"` |
| `metadata.working_directory` |
`"."` |
| `metadata.command` |
strip any absolute path prefix, keep as `"agentready assess ."` |
| `repository.path` |
`"."` |
Both `submit_with_gh_cli` and `submit_with_token` then serialize the sanitized dict rather than reading the raw file content.
The fix should include a unit test that asserts none of the above fields contain an absolute path or `@` character after sanitization.
Related
Bug Description
`agentready submit` commits the assessment JSON verbatim, including fields that contain local machine information:
This means every leaderboard submission leaks the submitter's username, hostname, and local directory layout. The submitter in PR #424 caught this only because CodeRabbit flagged it and manually redacted the fields in a follow-up commit.
Older submissions in `submissions/` still contain unredacted PII.
Root Cause
In `src/agentready/cli/submit.py`, both submission paths (`submit_with_gh_cli` lines 352-356 and `submit_with_token` lines 564-565) read the assessment file and commit it as-is:
```python
with open(assessment_path, encoding="utf-8") as f:
content = f.read()
```
There is no sanitization step.
Expected Behavior
`agentready submit` automatically strips local PII before committing the file to the leaderboard. Users should not need to manually review or redact their assessment before submitting.
Possible Solution
Add a `sanitize_for_submission(data: dict) -> dict` function in `submit.py` called right after `load_assessment()`. It should replace:
Both `submit_with_gh_cli` and `submit_with_token` then serialize the sanitized dict rather than reading the raw file content.
The fix should include a unit test that asserts none of the above fields contain an absolute path or `@` character after sanitization.
Related