Skip to content

Add model performance benchmark suite (time & peak RSS)#294

Open
allnes wants to merge 2 commits intoembedded-dev-research:mainfrom
allnes:an/python-benchmark
Open

Add model performance benchmark suite (time & peak RSS)#294
allnes wants to merge 2 commits intoembedded-dev-research:mainfrom
allnes:an/python-benchmark

Conversation

@allnes
Copy link
Copy Markdown
Member

@allnes allnes commented May 4, 2026

Adds a benchmark suite to measure Graph_Build performance across models and execution variants.

Files

  • benchmarks/model_performance.py
  • benchmarks/README.md
  • .gitignore update for benchmark result artifacts

Metrics

  • Wall time
  • Peak RSS memory (including child processes; psutil preferred, OS fallback otherwise)

Coverage

  • Models: alexnet_mnist, googlenet, densenet, resnet, yolo
  • Variants: seq, onednn, parallel-tbb, parallel-threads, parallel-omp, parallel-kokkos

Output

  • JSON (with environment metadata)
  • CSV (analysis/CI)

Run

cmake -S . -B build && cmake --build build --target Graph_Build --parallel

python3 benchmarks/model_performance.py \
  --model googlenet,resnet \
  --variant seq \
  --variant parallel-tbb \
  --repeat 3 \
  --warmup 1 \
  --csv-out benchmark_results/model_performance.csv

@allnes allnes requested a review from aobolensk as a code owner May 4, 2026 11:23
@codecov
Copy link
Copy Markdown

codecov Bot commented May 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.69%. Comparing base (60c1097) to head (e13836a).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #294   +/-   ##
=======================================
  Coverage   84.69%   84.69%           
=======================================
  Files          61       61           
  Lines        3744     3744           
  Branches     2218     2218           
=======================================
  Hits         3171     3171           
  Misses        285      285           
  Partials      288      288           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant