Skip to content

PERF: Optimize execute() hot path: soft reset, prepare caching, and guarded diagnostics #528

Draft
bewithgaurav wants to merge 1 commit intomainfrom
bewithgaurav/insertmany-perf-python-fixes
Draft

PERF: Optimize execute() hot path: soft reset, prepare caching, and guarded diagnostics #528
bewithgaurav wants to merge 1 commit intomainfrom
bewithgaurav/insertmany-perf-python-fixes

Conversation

@bewithgaurav
Copy link
Copy Markdown
Collaborator

@bewithgaurav bewithgaurav commented Apr 17, 2026

Work Item / Issue Reference

AB#44166

GitHub Issue: #<ISSUE_NUMBER>


Summary

This pull request introduces performance benchmarking infrastructure improvements and optimizations to the mssql_python driver, as well as documentation and workflow updates to support a new PERF: pull request prefix. The most significant changes are grouped below.

Driver Performance Optimizations:

  • Implemented a lightweight _soft_reset_cursor in cursor.py to reuse prepared statement handles, avoiding unnecessary SQLPrepare calls when executing the same SQL repeatedly. This improves performance for repeated statement execution. [1] [2] [3] [4]
  • Optimized parameter conversion logic to skip redundant conversions when re-executing the same SQL with the same parameter style, further reducing overhead.
  • Added a new DDBCSQLResetStmt binding in the C++ layer to support the lightweight reset operation, and exposed it to Python. [1] [2]

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 17, 2026

📊 Code Coverage Report

🔥 Diff Coverage

87%


🎯 Overall Coverage

79%


📈 Total Lines Covered: 6787 out of 8540
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

  • mssql_python/cursor.py (95.2%): Missing lines 1383
  • mssql_python/pybind/ddbc_bindings.cpp (77.8%): Missing lines 1383-1384,1387-1388

Summary

  • Total: 39 lines
  • Missing: 5 lines
  • Coverage: 87%

mssql_python/cursor.py

Lines 1379-1387

  1379         if reset_cursor:
  1380             if self.hstmt:
  1381                 self._soft_reset_cursor()
  1382             else:
! 1383                 self._reset_cursor()
  1384         else:
  1385             # Close just the ODBC cursor (not the statement handle) so the
  1386             # prepared plan can be reused.  SQLFreeStmt(SQL_CLOSE) releases
  1387             # the cursor associated with hstmt without destroying the

mssql_python/pybind/ddbc_bindings.cpp

Lines 1379-1392

  1379 }
  1380 
  1381 SQLRETURN SQLResetStmt_wrap(SqlHandlePtr statementHandle) {
  1382     if (!SQLFreeStmt_ptr) {
! 1383         DriverLoader::getInstance().loadDriver();
! 1384     }
  1385     SQLHANDLE hStmt = statementHandle->get();
  1386     if (!hStmt) {
! 1387         return SQL_INVALID_HANDLE;
! 1388     }
  1389 
  1390     SQLRETURN rc = SQLFreeStmt_ptr(hStmt, SQL_CLOSE);
  1391     if (SQL_SUCCEEDED(rc)) {
  1392         rc = SQLFreeStmt_ptr(hStmt, SQL_RESET_PARAMS);


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 67.8%
mssql_python.row.py: 70.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 74.6%
mssql_python.pybind.connection.connection.cpp: 75.8%
mssql_python.__init__.py: 77.3%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.connection.py: 85.3%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@bewithgaurav bewithgaurav force-pushed the bewithgaurav/insertmany-perf-python-fixes branch from ab57bb8 to 6703b85 Compare April 20, 2026 08:01
Comment thread eng/pipelines/pr-validation-pipeline.yml Dismissed
Comment thread eng/pipelines/pr-validation-pipeline.yml Dismissed
@bewithgaurav bewithgaurav force-pushed the bewithgaurav/insertmany-perf-python-fixes branch 2 times, most recently from 2ee0f68 to c113a03 Compare April 20, 2026 09:59
@github-actions github-actions Bot added the pr-size: large Substantial code update label Apr 20, 2026
@bewithgaurav bewithgaurav force-pushed the bewithgaurav/insertmany-perf-python-fixes branch 2 times, most recently from 5b92dc7 to 970ce4d Compare April 20, 2026 10:38
- Add _soft_reset_cursor: SQL_CLOSE + SQL_RESET_PARAMS instead of
  full HSTMT free/realloc on each execute() call
- Add DDBCSQLResetStmt C++ wrapper exposing lightweight reset via pybind11
- Skip SQLPrepare when re-executing the same SQL (prepare caching)
- Skip detect_and_convert_parameters on repeated same-SQL calls
- Guard DDBCSQLGetAllDiagRecords behind SQL_SUCCESS_WITH_INFO check
- Guard per-parameter debug logging behind logger.isEnabledFor(DEBUG)
- Fix benchmark script to strip Driver= from mssql-python connection string
@bewithgaurav bewithgaurav force-pushed the bewithgaurav/insertmany-perf-python-fixes branch from 970ce4d to 0604fc5 Compare April 24, 2026 11:13
@github-actions github-actions Bot added pr-size: medium Moderate update size and removed pr-size: large Substantial code update labels Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: medium Moderate update size

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants