Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions tools/sbom-diff-and-risk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ Offline `stale_package` evaluation is intentionally deferred. When enrichment is

- `report.json`
- `summary.json` when `--summary-json` is provided
- `policy.json` when `--policy-json` is provided
- `report.md`
- `report.sarif`

Expand Down Expand Up @@ -183,6 +184,7 @@ sbom-diff-risk compare \
- `--pyproject-group name`
- `--out-json path`
- `--summary-json path`
- `--policy-json path`
- `--out-md path`
- `--out-sarif path`
- `--policy path`
Expand All @@ -203,6 +205,12 @@ The checked-in [examples/sample-summary.json](examples/sample-summary.json) arti

For CI dashboard, job-summary, and local-threshold examples, see [docs/summary-json-ci-cookbook.md](docs/summary-json-ci-cookbook.md).

`--policy-json PATH` writes only policy-related JSON sections from the full
report. It includes `policy_evaluation`, policy finding lists, `rule_catalog`,
and `summary.policy` when policy evaluation is applied. For CI job-summary
examples, see
[docs/policy-decision-ci-cookbook.md](docs/policy-decision-ci-cookbook.md).

## Dependency Provenance Analysis (Opt-in)

This section is about analyzing third-party package provenance signals. It is not about verifying the `sbom-diff-and-risk` tool's own release artifacts.
Expand Down
22 changes: 12 additions & 10 deletions tools/sbom-diff-and-risk/docs/policy-decision-ci-cookbook.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Policy decision CI cookbook

This page shows how to consume policy decision explanation fields from
`report.json` in CI without changing the `sbom-diff-risk` analysis model.
This page shows how to consume policy decision explanation fields from the
`--policy-json PATH` sidecar in CI without changing the `sbom-diff-risk`
analysis model.

Use this when a repository wants a small job summary that explains local policy
blocks, warnings, or suppressions in machine-readable terms.
Expand All @@ -13,24 +14,25 @@ sbom-diff-risk compare \
--before examples/cdx_before.json \
--after examples/cdx_after.json \
--policy examples/policy-strict.yml \
--out-json outputs/policy-report.json
--out-json outputs/report.json \
--policy-json outputs/policy.json
```

The strict example policy can make the command return a policy failure exit
code. In CI, keep the generated `outputs/policy-report.json` artifact so the
policy decision metadata remains available for review.
code. In CI, keep the generated `outputs/policy.json` artifact so the policy
decision metadata remains available for review.

## Python consumer

This example reads the full JSON report, prints compact policy status, and then
prints the stable explanation fields for blocking and warning findings.
This example reads the policy-only JSON sidecar, prints compact policy status,
and then prints the stable explanation fields for blocking and warning findings.

```python
import json
from pathlib import Path

report = json.loads(
Path("outputs/policy-report.json").read_text(encoding="utf-8")
Path("outputs/policy.json").read_text(encoding="utf-8")
)

policy = report.get("summary", {}).get("policy")
Expand Down Expand Up @@ -73,10 +75,10 @@ tool. The snippet does not create a new package safety verdict.
## PowerShell consumer

This example uses `ConvertFrom-Json` to print the same policy status and
explanation fields.
explanation fields from the policy-only sidecar.

```powershell
$report = Get-Content outputs/policy-report.json -Raw | ConvertFrom-Json
$report = Get-Content outputs/policy.json -Raw | ConvertFrom-Json
$policy = $report.summary.policy

if ($null -eq $policy) {
Expand Down
12 changes: 12 additions & 0 deletions tools/sbom-diff-and-risk/docs/report-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,18 @@ For reviewer-facing examples and interpretation guidance, see
consumer snippets, see
[policy-decision-ci-cookbook.md](policy-decision-ci-cookbook.md).

The `--policy-json PATH` CLI option writes a policy-only JSON sidecar using the
same policy-related sections from the full JSON report:

- `policy_evaluation`
- `blocking_findings`
- `warning_findings`
- `suppressed_findings`
- `rule_catalog`
- `summary.policy` when policy evaluation is applied
- `provenance_policy` and `provenance_policy_impact` when provenance policy
fields are relevant

## Summary contract

`summary` is the stable, compact entry point for automation that needs counts
Expand Down
40 changes: 34 additions & 6 deletions tools/sbom-diff-and-risk/src/sbom_diff_risk/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from .policy_evaluator import evaluate_policy
from .policy_parser import build_policy
from .presentation import effective_policy_evaluation, summarize_violations_by_rule
from .report_json import render_report_json, render_summary_json
from .report_json import render_policy_json, render_report_json, render_summary_json
from .report_md import render_report_markdown
from .report_sarif import render_report_sarif_output
from .risk import evaluate_risks, summarize_risks
Expand All @@ -31,7 +31,10 @@ def build_parser() -> argparse.ArgumentParser:
"compare",
help="Compare two dependency inputs and write JSON and/or Markdown reports.",
description="Compare two local dependency inputs and emit deterministic reports.",
epilog="Exit codes: 0 = success/no blocking violations, 1 = blocking policy violations, 2 = usage/parse/runtime error.",
epilog=(
"Exit codes: 0 = success/no blocking violations, "
"1 = blocking policy violations, 2 = usage/parse/runtime error."
),
)
compare.add_argument("--before", type=Path, required=True, help="Path to the before input.")
compare.add_argument("--after", type=Path, required=True, help="Path to the after input.")
Expand Down Expand Up @@ -59,7 +62,18 @@ def build_parser() -> argparse.ArgumentParser:
help="Select a PEP 735 [dependency-groups] group when a compared input is pyproject.toml.",
)
compare.add_argument("--out-json", type=Path, default=None, help="Write a JSON report to this path.")
compare.add_argument("--summary-json", type=Path, default=None, help="Write the stable JSON summary object to this path.")
compare.add_argument(
"--summary-json",
type=Path,
default=None,
help="Write the stable JSON summary object to this path.",
)
compare.add_argument(
"--policy-json",
type=Path,
default=None,
help="Write policy evaluation and policy finding JSON sections to this path.",
)
compare.add_argument("--out-md", type=Path, default=None, help="Write a Markdown report to this path.")
compare.add_argument(
"--out-sarif",
Expand Down Expand Up @@ -91,7 +105,10 @@ def build_parser() -> argparse.ArgumentParser:
compare.add_argument(
"--enrich-pypi",
action="store_true",
help="Opt-in PyPI provenance and integrity enrichment. Default behavior remains offline with no network access.",
help=(
"Opt-in PyPI provenance and integrity enrichment. "
"Default behavior remains offline with no network access."
),
)
compare.add_argument(
"--pypi-timeout",
Expand Down Expand Up @@ -139,8 +156,17 @@ def run_compare(args: argparse.Namespace) -> int:
scorecard_timeout = getattr(args, "scorecard_timeout", DEFAULT_SCORECARD_TIMEOUT_SECONDS)

summary_json = getattr(args, "summary_json", None)
if args.out_json is None and summary_json is None and args.out_md is None and args.out_sarif is None:
raise ValueError("at least one of --out-json, --summary-json, --out-md, or --out-sarif must be provided")
policy_json = getattr(args, "policy_json", None)
if (
args.out_json is None
and summary_json is None
and policy_json is None
and args.out_md is None
and args.out_sarif is None
):
raise ValueError(
"at least one of --out-json, --summary-json, --policy-json, --out-md, or --out-sarif must be provided"
)
if pypi_timeout <= 0:
raise ValueError("--pypi-timeout must be a positive number of seconds.")
if scorecard_timeout <= 0:
Expand Down Expand Up @@ -232,6 +258,8 @@ def run_compare(args: argparse.Namespace) -> int:
_write_text(args.out_json, render_report_json(report))
if summary_json is not None:
_write_text(summary_json, render_summary_json(report))
if policy_json is not None:
_write_text(policy_json, render_policy_json(report))
if args.out_md is not None:
_write_text(args.out_md, render_report_markdown(report))
if args.out_sarif is not None:
Expand Down
21 changes: 21 additions & 0 deletions tools/sbom-diff-and-risk/src/sbom_diff_risk/report_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,27 @@ def render_summary_json(report: CompareReport) -> str:
return json.dumps(_summary_to_dict(report), indent=2) + "\n"


def render_policy_json(report: CompareReport) -> str:
policy_sections = build_policy_report_sections(report.metadata.policy_evaluation)
payload: dict[str, object] = {
"policy_evaluation": policy_sections["policy_evaluation"],
"blocking_findings": policy_sections["blocking_findings"],
"warning_findings": policy_sections["warning_findings"],
"suppressed_findings": policy_sections["suppressed_findings"],
"rule_catalog": policy_sections["rule_catalog"],
}

policy_summary = _policy_summary_to_dict(report.metadata.policy_evaluation)
if policy_summary is not None:
payload["summary"] = {"policy": policy_summary}

if policy_sections["provenance_policy"] is not None:
payload["provenance_policy"] = policy_sections["provenance_policy"]
payload["provenance_policy_impact"] = policy_sections["provenance_policy_impact"]

return json.dumps(payload, indent=2) + "\n"


def _summary_to_dict(report: CompareReport) -> dict[str, object]:
summary: dict[str, object] = {
"added": report.summary.added,
Expand Down
1 change: 1 addition & 0 deletions tools/sbom-diff-and-risk/tests/test_cli_exit_codes.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ def test_cli_compare_help_mentions_policy_flags_and_exit_codes() -> None:
assert result.returncode == 0
assert "--out-sarif" in result.stdout
assert "--summary-json" in result.stdout
assert "--policy-json" in result.stdout
assert "--pyproject-group" in result.stdout
assert "--policy" in result.stdout
assert "--fail-on" in result.stdout
Expand Down
142 changes: 142 additions & 0 deletions tools/sbom-diff-and-risk/tests/test_cli_policy_json.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
from __future__ import annotations

import json
from pathlib import Path

from sbom_diff_risk import cli


def test_cli_policy_json_writes_policy_only_file(tmp_path: Path) -> None:
project_root = Path(__file__).resolve().parents[1]
policy_path = tmp_path / "policy.json"

exit_code = cli.main(
[
"compare",
"--before",
str(project_root / "examples" / "cdx_before.json"),
"--after",
str(project_root / "examples" / "cdx_after.json"),
"--policy",
str(project_root / "examples" / "policy-strict.yml"),
"--policy-json",
str(policy_path),
]
)

payload = json.loads(policy_path.read_text(encoding="utf-8"))

assert exit_code == 1
assert payload["summary"]["policy"] == {
"status": "fail",
"blocking": 3,
"warning": 1,
"suppressed": 0,
}
assert payload["policy_evaluation"]["applied"] is True
assert payload["policy_evaluation"]["exit_code"] == 1
assert len(payload["blocking_findings"]) == 3
assert len(payload["warning_findings"]) == 1
assert payload["blocking_findings"][0]["decision_reason"] == "added_package_count_exceeded_threshold"
assert payload["blocking_findings"][0]["policy_rule"] == "max_added_packages"
assert "components" not in payload
assert "risks" not in payload
assert policy_path.read_text(encoding="utf-8").endswith("\n")


def test_cli_policy_json_matches_full_report_policy_sections(tmp_path: Path) -> None:
project_root = Path(__file__).resolve().parents[1]
report_path = tmp_path / "report.json"
policy_path = tmp_path / "policy.json"

exit_code = cli.main(
[
"compare",
"--before",
str(project_root / "examples" / "cdx_before.json"),
"--after",
str(project_root / "examples" / "cdx_after.json"),
"--policy",
str(project_root / "examples" / "policy-strict.yml"),
"--out-json",
str(report_path),
"--policy-json",
str(policy_path),
]
)

report_payload = json.loads(report_path.read_text(encoding="utf-8"))
policy_payload = json.loads(policy_path.read_text(encoding="utf-8"))

assert exit_code == 1
assert policy_payload == _policy_sidecar_from_full_report(report_payload)


def test_cli_policy_json_without_policy_records_not_applied(tmp_path: Path) -> None:
project_root = Path(__file__).resolve().parents[1]
policy_path = tmp_path / "policy.json"

exit_code = cli.main(
[
"compare",
"--before",
str(project_root / "examples" / "cdx_before.json"),
"--after",
str(project_root / "examples" / "cdx_after.json"),
"--policy-json",
str(policy_path),
]
)

payload = json.loads(policy_path.read_text(encoding="utf-8"))

assert exit_code == 0
assert payload["policy_evaluation"]["applied"] is False
assert payload["policy_evaluation"]["exit_code"] == 0
assert "summary" not in payload
assert payload["blocking_findings"] == []
assert payload["warning_findings"] == []
assert payload["suppressed_findings"] == []


def test_cli_policy_json_omitted_does_not_write_policy_file(tmp_path: Path) -> None:
project_root = Path(__file__).resolve().parents[1]
report_path = tmp_path / "report.json"
policy_path = tmp_path / "policy.json"

exit_code = cli.main(
[
"compare",
"--before",
str(project_root / "examples" / "cdx_before.json"),
"--after",
str(project_root / "examples" / "cdx_after.json"),
"--out-json",
str(report_path),
]
)

assert exit_code == 0
assert report_path.is_file()
assert not policy_path.exists()


def _policy_sidecar_from_full_report(report_payload: dict[str, object]) -> dict[str, object]:
policy_payload = {
"policy_evaluation": report_payload["policy_evaluation"],
"blocking_findings": report_payload["blocking_findings"],
"warning_findings": report_payload["warning_findings"],
"suppressed_findings": report_payload["suppressed_findings"],
"rule_catalog": report_payload["rule_catalog"],
}

summary = report_payload["summary"]
assert isinstance(summary, dict)
if "policy" in summary:
policy_payload["summary"] = {"policy": summary["policy"]}

if "provenance_policy" in report_payload:
policy_payload["provenance_policy"] = report_payload["provenance_policy"]
policy_payload["provenance_policy_impact"] = report_payload["provenance_policy_impact"]

return policy_payload