Skip to content

Add Conley (1999) spatial HAC SE on DiD/TWFE/MultiPeriodDiD (Phase 1 of spillover-conley)#411

Open
igerber wants to merge 8 commits intomainfrom
spillover-conley
Open

Add Conley (1999) spatial HAC SE on DiD/TWFE/MultiPeriodDiD (Phase 1 of spillover-conley)#411
igerber wants to merge 8 commits intomainfrom
spillover-conley

Conversation

@igerber
Copy link
Copy Markdown
Owner

@igerber igerber commented May 10, 2026

Summary

  • Add vcov_type="conley" on DifferenceInDifferences, TwoWayFixedEffects, and MultiPeriodDiD with four new __init__ kwargs: conley_coords, conley_cutoff_km, conley_metric, conley_kernel. Variance estimator is Conley (1999) Eq 4.2 in pairwise form: Var̂(β) = (X'X)^{-1} · (Σ_{i,j} K(d_ij/h) · X_i ε_i ε_j X_j') · (X'X)^{-1}. Helpers live in new diff_diff/conley.py module (kept separate from linalg.py so the linear-algebra backend stays focused on OLS/sandwich primitives).
  • FWL composes cleanly because the meat depends only on scores X·ε, both of which within-transformation preserves. TwoWayFixedEffects(vcov_type="conley", ...) is therefore supported (unlike hc2/hc2_bm which need the full hat matrix). TWFE auto-cluster-at-unit is disabled when vcov_type="conley"; explicit cluster=, survey_design=, absorb=, or inference="wild_bootstrap" each raise NotImplementedError (deferred to Phase 2+ follow-ups). SyntheticDiD(vcov_type="conley") raises TypeError at both __init__ and set_params (uses bootstrap variance, not the analytical sandwich).
  • R conleyreg parity (Düsterhöft 2021, CRAN v0.1.9) on three benchmark fixtures at benchmarks/data/r_conleyreg_conley_golden.json; observed max abs diff 5.7e-16. Earth radius 6371.01 km matches conleyreg::haversine_dist. Regenerate via cd benchmarks/R && Rscript generate_conley_golden.R. Test file tests/test_conley_vcov.py skips parity cleanly when the JSON is absent (per the feedback_golden_file_pytest_skip convention; CI's isolated-install job ships only tests/).

Methodology references (required if estimator / math changes)

  • Method name(s): Conley spatial HAC standard errors (one-way, cross-sectional). New REGISTRY section ## ConleySpatialHAC.
  • Paper / source link(s):
    • Conley, T. G. (1999). "GMM Estimation with Cross-Sectional Dependence." Journal of Econometrics 92(1), 1-45. DOI: 10.1016/S0304-4076(98)00084-0
    • Andrews, D. W. K. (1991). "Heteroskedasticity and autocorrelation consistent covariance matrix estimation." Econometrica 59(3), 817-858.
    • Düsterhöft, C. (2021). conleyreg: Estimations using Conley Standard Errors. CRAN R package, https://github.com/cdueben/conleyreg. Parity benchmark target.
    • Colella, F., Lalive, R., Sakalli, S. O., & Thoenig, M. (2019). "Inference with Arbitrary Clustering." IZA DP No. 12584. Stata acreg reference; cited but not parity-tested here (Stata is paid-only).
    • Five paper-review markdowns under docs/methodology/papers/ (Conley 1999, Colella 2019, Clarke 2017, Butts 2023, Butts 2021) committed as the methodology basis for ConleySpatialHAC and the rescoped Phase 3 ring-indicator design tracked in BRIEFING.md.
  • Any intentional deviations from the source (and why): None for the Conley variance computation itself. The Earth radius constant (6371.01 km) was chosen to match R conleyreg exactly; the methodologically-equivalent 6371.0 km was the original draft value but produces ~1.5e-6 relative drift vs conleyreg and would break the 1e-6 parity bound. All deferrals (cluster, weights, absorb, wild_bootstrap, SyntheticDiD) are documented as NotImplementedError/TypeError raises with explicit pointers to the deferral phase, per feedback_no_silent_failures.

Validation

  • Tests added/updated: tests/test_conley_vcov.py (new; 74 tests across 12 test classes covering kernels, distance metrics, validator helpers, dispatch parity, estimator integration on DiD/MultiPeriodDiD/TWFE, set_params atomicity, defensive SyntheticDiD rejection, TWFE Conley+cluster/wild_bootstrap rejection, and R conleyreg parity at atol=1e-6 on three haversine fixtures).
  • Backtest / simulation / notebook evidence (if applicable): R parity fixtures (3) committed at benchmarks/data/r_conleyreg_conley_golden.json, observed max abs diff 5.7e-16. No tutorial added in Phase 1; the practitioner-facing T22 tutorial demonstrating Phase 3 ring-indicator spillover-aware DiD with Conley SE on top is queued for Phase 3 of the initiative.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes (verified by /pre-merge-check Section 2.6 secret-pattern scan; only synthetic test data and generated fixtures).

🤖 Generated with Claude Code

@github-actions
Copy link
Copy Markdown

PR Review: Conley Spatial HAC

Overall Assessment: ⚠️ Needs changes

Executive Summary

  • Affected methods: ConleySpatialHAC via DifferenceInDifferences, TwoWayFixedEffects, MultiPeriodDiD, and compute_robust_vcov.
  • The core helper implements the documented pairwise spatial sandwich, but panel estimators feed all unit-time rows into a spatial-only kernel with no time key, lag cutoff, or front-door rejection.
  • Unsupported combinations like Conley + cluster/survey/absorb are mostly rejected consistently; SyntheticDiD rejection is mirrored in __init__ and set_params.
  • Result/reporting propagation is incomplete: conley_metric is used for estimation but dropped from result objects, causing summaries to hard-code km even for euclidean/custom metrics.
  • Pattern-wide inference grep found no new Conley inline t_stat = effect / se anti-pattern; Conley analytical paths route through safe_inference.

Methodology

P1 — Panel estimators silently use cross-sectional Conley over all unit-time rows

Locations: diff_diff/conley.py:L216-L239, diff_diff/estimators.py:L484-L532, diff_diff/estimators.py:L1607-L1630, diff_diff/twfe.py:L373-L416, docs/methodology/REGISTRY.md:L2927-L2936, docs/methodology/papers/conley-1999-review.md:L226-L230, docs/methodology/papers/conley-1999-review.md:L273-L277, benchmarks/R/generate_conley_golden.R:L33-L50

Impact: The registry and paper review define Phase 1 as cross-sectional spatial HAC, with the time dimension deferred to Phase 2. The implementation nevertheless passes every long-panel row into _compute_conley_vcov; the kernel is only K(d_ij / h), so repeated observations for the same unit across periods have d_ij = 0 and receive full covariance weight, and all cross-time pairs are weighted only by spatial distance. That is neither the cross-sectional Conley fixture validated against R (time <- 1L, lag_cutoff = 0) nor a documented space-time/product-kernel estimator. This can silently misstate SEs on DiD/TWFE/MultiPeriod panels.

Concrete fix: Either reject vcov_type="conley" on long-panel estimator fits until Phase 2, or implement an explicit space-time HAC contract with a time key and lag cutoff/product kernel. Add regression tests where repeated coordinates appear across multiple periods and assert either a clear rejection or parity against a validated panel/space-time reference.

Code Quality

P1 — conley_metric is dropped at the result/reporting boundary

Locations: diff_diff/results.py:L49-L82, diff_diff/results.py:L140-L142, diff_diff/results.py:L219-L226, diff_diff/results.py:L461-L463, diff_diff/results.py:L561-L568, diff_diff/estimators.py:L650-L656, diff_diff/estimators.py:L1913-L1921, diff_diff/twfe.py:L580-L585

Impact: conley_metric is a new constructor parameter and affects the variance calculation, but DiDResults / MultiPeriodDiDResults only retain cutoff and kernel. _format_vcov_label() therefore always prints cutoff=...km, which is wrong for conley_metric="euclidean" or a callable metric. This violates the new-parameter downstream propagation rule and can mislead users reading summaries.

Concrete fix: Add conley_metric and preferably conley_coords metadata to the result dataclasses, pass them from DiD/TWFE/MultiPeriod result construction, and update _format_vcov_label() to show km only for haversine and neutral coordinate/custom units otherwise. Add summary tests for euclidean and callable metric paths.

P3 — Callable metric validation is incomplete but tracked

Locations: diff_diff/conley.py:L81-L82, TODO.md:L120

Impact: A malformed callable can return non-square, non-finite, negative, or asymmetric distances and fail later with opaque errors or asymmetric vcov output.

Concrete fix: Already tracked in TODO.md; validate callable output shape, finiteness, non-negativity, and symmetry before forming the kernel.

Performance

No unmitigated findings. The dense O(n^2) distance matrix path warns at large n in diff_diff/conley.py:L174-L181, and the sparse fast path is documented as deferred in docs/methodology/REGISTRY.md:L2935-L2936.

Maintainability

P3 — Conley estimator-level validation is duplicated but tracked

Locations: diff_diff/estimators.py:L379-L417, diff_diff/estimators.py:L1431-L1473, diff_diff/twfe.py:L181-L209, TODO.md:L121

Impact: Future Conley support changes could update one estimator surface but miss another.

Concrete fix: Already tracked in TODO.md; extract a shared estimator-level Conley validation helper.

Tech Debt

The PR adds TODO entries for deferred Conley + cluster, survey weights, absorb, SyntheticDiD, callable metric validation, validation deduplication, and stronger FWL testing at TODO.md:L116-L122. These are acceptable as P3/tracked items except where they intersect the unmitigated P1 findings above.

Security

No findings. Secret-pattern sweep over the changed tree, excluding the requested noisy paths, found no matches.

Documentation/Tests

P3 — TWFE FWL test name overstates its assertion, tracked

Locations: tests/test_conley_vcov.py:L817-L839, TODO.md:L122

Impact: The test claims FWL invariance but only checks that TWFE and simple DiD Conley SEs are finite. This does not validate equivalence to a full-dummy FE design.

Concrete fix: Already tracked in TODO.md; add a full-dummy regression comparison for ATT and Conley SE.

Path to Approval

  1. Add a front-door guard rejecting Conley on multi-period/long-panel estimator fits, or implement a documented space-time HAC with time key and lag cutoff/product kernel.
  2. Add tests for repeated coordinates across periods on TWFE and MultiPeriodDiD, asserting rejection or parity to a validated panel reference.
  3. Propagate conley_metric and Conley coordinate metadata into result objects and fix summary labeling for non-haversine metrics.

igerber added a commit that referenced this pull request May 10, 2026
…ring

Address CI Codex review of PR #411 (P1#1 + P1#2):

P1#1 — Panel estimators with vcov_type="conley" silently produced wrong
SE because cross-sectional Conley over (unit, time) rows treated same-
unit cross-time pairs as d_ij=0 -> K=1, mishandling the space-time HAC.
Phase 1 supports cross-sectional Conley only; reject panel fits at
fit-time on DifferenceInDifferences, TwoWayFixedEffects, and
MultiPeriodDiD with NotImplementedError. Practitioners pre-collapse to
per-unit first-differences and call compute_robust_vcov directly. Phase
2 will add the space-time product kernel (Driscoll-Kraay) and lift the
rejection. Granular Conley-arg validation collapsed into the single
unconditional reject (cluster/absorb/coords/cutoff combinations all
hit the same path).

P1#2 — conley_metric was dropped at the result boundary and
_format_vcov_label hard-coded "km" for the cutoff label even when
metric was "euclidean". With panels rejected, the conley_cutoff_km /
conley_kernel fields on DiDResults / MultiPeriodDiDResults are now
unreachable; remove the dead fields, the dead arg passes from
estimators.py / twfe.py, and the dead "conley" branch in
_format_vcov_label.

Tests added: TWFE / DiD / MPD panel-rejection regressions, including a
repeated-coords-across-periods regression per the CI reviewer's
recommendation. 70 Conley tests + 401 targeted regression tests pass.

REGISTRY / CHANGELOG / llms.txt / README / TODO updated to reflect
that the only supported Phase 1 Conley path is direct
LinearRegression / compute_robust_vcov on a single-period design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 37288311afe45064a9deeede9b8bfee8662dc781


PR Review: Conley Spatial HAC

Overall Assessment: ⚠️ Needs changes

Executive Summary

  • Previous P1 methodology finding is still open: panel estimators still run spatial-only Conley over (unit, time) rows.
  • Previous P1 reporting finding is still open: conley_metric affects computation but is dropped from result metadata and summaries.
  • Core cross-sectional helper follows the documented pairwise sandwich and rejects cluster/weights at the linalg layer.
  • SyntheticDiD rejection is mirrored in __init__ and set_params.
  • Pattern-wide inference grep found no new inline t_stat = effect / se anti-pattern in affected Conley paths.
  • I could not run pytest in this environment because numpy is unavailable.

Methodology

P1 — Panel estimators still silently use cross-sectional Conley over long-panel rows

Locations: diff_diff/estimators.py:L379-L417, diff_diff/estimators.py:L484-L532, diff_diff/estimators.py:L1440-L1473, diff_diff/estimators.py:L1611-L1630, diff_diff/twfe.py:L181-L208, diff_diff/twfe.py:L373-L415, diff_diff/conley.py:L216-L239, docs/methodology/REGISTRY.md:L2927-L2936, docs/methodology/REGISTRY.md:L2962-L2968, docs/methodology/papers/conley-1999-review.md:L16-L20, docs/methodology/papers/conley-1999-review.md:L273-L277

Impact: DifferenceInDifferences, MultiPeriodDiD, and TwoWayFixedEffects reject Conley only for cluster/survey/absorb/missing-parameter combinations, then pass every panel row into _compute_conley_vcov. The kernel is only K(d_ij / h), so repeated unit coordinates across periods get d_ij = 0 and full covariance weight, with no time key or lag cutoff. The registry says Phase 1 is cross-sectional and Phase 2 adds the time dimension, while a separate FWL note claims TWFE support; that FWL note does not document or justify the missing space-time HAC contract. This can silently misstate SEs.

Concrete fix: Either reject vcov_type="conley" on DifferenceInDifferences, TwoWayFixedEffects, and MultiPeriodDiD until Phase 2, or implement an explicit space-time HAC with a time key and lag cutoff/product kernel. Add repeated-coordinate panel tests asserting rejection or parity against a validated space-time reference.

Code Quality

P1 — conley_metric is still dropped at the result/reporting boundary

Locations: diff_diff/results.py:L49-L82, diff_diff/results.py:L140-L142, diff_diff/results.py:L461-L463, diff_diff/estimators.py:L653-L656, diff_diff/estimators.py:L1920-L1921, diff_diff/twfe.py:L584-L585

Impact: conley_metric is used in estimation, but result classes only store cutoff and kernel. _format_vcov_label() always renders cutoff=...km, which is wrong for conley_metric="euclidean" or a callable metric. This is incomplete propagation of a new public parameter and can mislead users.

Concrete fix: Add conley_metric to DiDResults / MultiPeriodDiDResults, pass it from DiD/TWFE/MultiPeriod constructors, and render units as km only for haversine. Add summary tests for euclidean and callable metrics. If panel Conley is rejected instead, remove this unreachable panel-result Conley plumbing/tests.

P3 — Callable metric validation is incomplete but tracked

Locations: diff_diff/conley.py:L81-L82, TODO.md:L120

Impact: A malformed callable can return non-square, non-finite, negative, or asymmetric distances and fail later with opaque errors.

Concrete fix: Already tracked in TODO.md; validate callable output shape, finiteness, non-negativity, and symmetry.

Performance

No unmitigated findings. Dense O(n^2) distance-matrix warning is present in diff_diff/conley.py:L174-L181, and sparse fast path is documented as deferred.

Maintainability

P3 — Conley estimator validation is duplicated but tracked

Locations: diff_diff/estimators.py:L379-L417, diff_diff/estimators.py:L1440-L1473, diff_diff/twfe.py:L181-L208, TODO.md:L121

Impact: Future Conley support changes can update one estimator surface but miss another.

Concrete fix: Already tracked in TODO.md; extract a shared estimator-level Conley validation helper.

Tech Debt

The Conley callable-metric validation, validation deduplication, and stronger TWFE FWL test are tracked in TODO.md:L120-L122, so I classify those as P3 informational. The panel-space-time methodology issue is not mitigated by TODO because it is an unmitigated P1 methodology/SE issue.

Security

No findings. Secret-pattern scan over changed areas found only benign references such as comments/tests mentioning tokens or /etc/passwd.

Documentation/Tests

P1 — Tests and user docs currently lock in the unsupported panel behavior

Locations: tests/test_conley_vcov.py:L663-L743, tests/test_conley_vcov.py:L746-L839, docs/methodology/REGISTRY.md:L2930-L2936, diff_diff/guides/llms-full.txt:L1886-L1904, diff_diff/guides/llms.txt:L78-L79, README.md:L127

Impact: The tests assert that DiD/MultiPeriodDiD/TWFE with Conley produce finite SEs, and docs advertise panel support. That reinforces the P1 methodology issue rather than guarding against it.

Concrete fix: If Phase 1 remains cross-sectional, change these tests to expect NotImplementedError for panel estimators and update README/CHANGELOG/LLM guides/REGISTRY accordingly. If panel support is retained, add validated space-time reference tests.

P3 — TWFE FWL test name overstates its assertion, tracked

Locations: tests/test_conley_vcov.py:L817-L839, TODO.md:L122

Impact: The test claims FWL invariance but only checks finite SEs from TWFE and simple DiD.

Concrete fix: Already tracked in TODO.md; compare against an explicit full-dummy FE design.

Audit Confirmation

Sibling-surface audit covered DiD ↔ MultiPeriodDiD ↔ TWFE ↔ SyntheticDiD, result schema ↔ renderer, and docs ↔ tests. Pattern-wide inference grep found no new Conley inline inference anti-pattern. Reciprocal guard checks covered cluster, survey/weights, absorb, and wild-bootstrap directions. No changed GH Actions workflow path selectors were in scope.

Path to Approval

  1. Choose one Conley panel contract:
    • Phase-1 route: add front-door NotImplementedError for DifferenceInDifferences, TwoWayFixedEffects, and MultiPeriodDiD, and update tests/docs to state only LinearRegression / compute_robust_vcov are supported.
    • Full-support route: implement a documented space-time HAC with time key and lag cutoff/product kernel, plus parity/regression tests.
  2. Add repeated-coordinate panel tests for DiD, TWFE, and MultiPeriodDiD that assert rejection or validated space-time parity.
  3. Propagate conley_metric into result metadata and summary labels, or remove panel-result Conley reporting if panel support is rejected.

igerber added a commit that referenced this pull request May 10, 2026
…ring

Address CI Codex review of PR #411 (P1#1 + P1#2):

P1#1 — Panel estimators with vcov_type="conley" silently produced wrong
SE because cross-sectional Conley over (unit, time) rows treated same-
unit cross-time pairs as d_ij=0 -> K=1, mishandling the space-time HAC.
Phase 1 supports cross-sectional Conley only; reject panel fits at
fit-time on DifferenceInDifferences, TwoWayFixedEffects, and
MultiPeriodDiD with NotImplementedError. Practitioners pre-collapse to
per-unit first-differences and call compute_robust_vcov directly. Phase
2 will add the space-time product kernel (Driscoll-Kraay) and lift the
rejection. Granular Conley-arg validation collapsed into the single
unconditional reject (cluster/absorb/coords/cutoff combinations all
hit the same path).

P1#2 — conley_metric was dropped at the result boundary and
_format_vcov_label hard-coded "km" for the cutoff label even when
metric was "euclidean". With panels rejected, the conley_cutoff_km /
conley_kernel fields on DiDResults / MultiPeriodDiDResults are now
unreachable; remove the dead fields, the dead arg passes from
estimators.py / twfe.py, and the dead "conley" branch in
_format_vcov_label.

Tests added: TWFE / DiD / MPD panel-rejection regressions, including a
repeated-coords-across-periods regression per the CI reviewer's
recommendation. 70 Conley tests + 401 targeted regression tests pass.

REGISTRY / CHANGELOG / llms.txt / README / TODO updated to reflect
that the only supported Phase 1 Conley path is direct
LinearRegression / compute_robust_vcov on a single-period design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber igerber force-pushed the spillover-conley branch from 3728831 to 8be2ebb Compare May 10, 2026 18:08
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 8be2ebb359763141606954b0685092c99f315c5d


PR Review: Conley Spatial HAC

Overall Assessment: ⚠️ Needs changes

Executive Summary

  • The previous P1 panel-methodology issue is resolved: DiD, TWFE, and MultiPeriodDiD now reject vcov_type="conley" at fit time, matching the registry.
  • The previous conley_metric result-label issue is effectively resolved because panel result Conley output is now unreachable.
  • One P1 remains: LinearRegression(vcov_type="conley", survey_design=...) can silently compute survey variance instead of raising the documented Conley+survey NotImplementedError.
  • llms-full.txt and some estimator docstrings still advertise panel Conley behavior that now raises.
  • I could not run tests in this environment because numpy and pytest are unavailable.

Methodology

P1 — LinearRegression silently bypasses the documented Conley+survey rejection

Locations: diff_diff/linalg.py:L2516-L2586, diff_diff/linalg.py:L2700-L2773, diff_diff/linalg.py:L1080-L1097, docs/methodology/REGISTRY.md:L2994-L2999, TODO.md:L116-L120

Impact: The registry says cross-sectional LinearRegression / compute_robust_vcov with vcov_type="conley" plus weights= / survey_design= must raise NotImplementedError. compute_robust_vcov and weighted solve_ols hit _validate_vcov_args, but LinearRegression.fit() sets return_vcov=not _use_survey_vcov; with a resolved survey design, Conley validation is skipped and compute_survey_vcov() overwrites vcov while _fit_vcov_type_ remains "conley". This silently returns a survey variance under a Conley request.

Concrete fix: In LinearRegression.fit(), after resolving _fit_vcov_type and survey design, raise NotImplementedError when _fit_vcov_type == "conley" and self.survey_design is not None / _use_survey_vcov is true. Add a regression test using make_pweight_design(np.ones(n)) with LinearRegression(vcov_type="conley", conley_coords=..., conley_cutoff_km=..., survey_design=...).

Code Quality

No unmitigated code-quality findings beyond the missing guard above.

Performance

No unmitigated findings. The dense O(n^2) Conley distance-matrix warning and sparse fast-path deferral are documented/tracked.

Maintainability

P3 — Callable metric validation remains deferred and tracked

Locations: diff_diff/conley.py:L66-L82, TODO.md:L120

Impact: A malformed callable metric can return non-square, non-finite, negative, or asymmetric distances and fail later with opaque errors.

Concrete fix: Already tracked in TODO.md; validate callable output shape, finiteness, non-negativity, and symmetry before building the kernel.

Tech Debt

Tracked Conley deferrals in TODO.md are acceptable as P3 informational items where the code fails fast. The survey bypass above is not mitigated by TODO because the tracked contract says it currently raises, but the LinearRegression survey path does not.

Security

No security findings. A targeted secret-pattern sweep over changed areas found only benign references in .gitignore and changelog text.

Documentation/Tests

P2 — User docs still show unsupported panel Conley

Locations: diff_diff/guides/llms-full.txt:L1886-L1927, diff_diff/estimators.py:L60-L76, diff_diff/estimators.py:L1039-L1053

Impact: llms-full.txt says Conley is available on DifferenceInDifferences, TwoWayFixedEffects, and MultiPeriodDiD, and shows a TWFE example that now raises. The estimator docstrings also describe Conley as if coords/cutoff validation is reachable on these panel estimators, while fit-time rejects all panel Conley first.

Concrete fix: Rewrite the Conley guide section around LinearRegression / compute_robust_vcov; explicitly state DiD/TWFE/MultiPeriod reject until Phase 2. Update the DiD and MultiPeriodDiD docstrings to match the registry’s Phase 1 contract.

Test execution

Not run: python -c "import numpy" fails with ModuleNotFoundError: No module named 'numpy', and pytest is not installed.

Audit Confirmation

Sibling-surface audit covered DiD ↔ MultiPeriodDiD ↔ TWFE ↔ SyntheticDiD, compute_robust_vcovsolve_olsLinearRegression, result schema ↔ renderer, and README/LLM guides/REGISTRY/TODO/tests. Pattern-wide inference grep found no new Conley inline t_stat = effect / se anti-pattern. Reciprocal guards covered cluster, weights/survey, panel rejection, and SyntheticDiD __init__/set_params. No changed GH Actions workflow path selectors were in scope.

Path to Approval

  1. Add the LinearRegression.fit() guard for vcov_type="conley" with any resolved survey_design, raising NotImplementedError.
  2. Add a regression test proving LinearRegression(vcov_type="conley", survey_design=make_pweight_design(...)) raises before fitting or setting vcov_.
  3. Update diff_diff/guides/llms-full.txt and panel estimator docstrings so they advertise only cross-sectional Conley support in Phase 1.

igerber added a commit that referenced this pull request May 10, 2026
Address CI Codex review of PR #411 (P1 + P2):

P1 — LinearRegression(vcov_type="conley", survey_design=...) silently
bypassed the documented Conley+survey rejection. The downstream
_validate_vcov_args (inside compute_robust_vcov) rejects this combination,
but LinearRegression.fit() sets return_vcov=False on the solve_ols call
when survey vcov is needed (so the linalg validator never runs), and the
survey vcov path then overwrites vcov_ with a non-Conley variance under a
Conley request. Front-door the rejection at LinearRegression entry so the
contract is enforced uniformly. New regression test
test_linear_regression_conley_with_survey_design_raises locks the new
guard with a make_pweight_design fixture.

P2 — DifferenceInDifferences and MultiPeriodDiD docstrings still listed
"conley" as a vcov_type option and described the conley_* kwargs as if
the path were reachable. Updated both to spell out the Phase 1 panel
rejection and point users at compute_robust_vcov / LinearRegression for
cross-sectional Conley. Rewrote the llms-full.txt Conley section around
the LinearRegression / compute_robust_vcov surface, replaced the panel-
estimator example with a cross-sectional one, and listed the panel
rejection in the restrictions table.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 1cb14d7cd73a9febe0d2dd82686e68905b66314d


PR Review: Conley Spatial HAC Re-review

Overall Assessment: ⚠️ Needs changes

Executive Summary

  • The prior P1 is resolved: LinearRegression(vcov_type="conley", survey_design=...) now raises before fitting, with a regression test.
  • Panel rejection is mirrored across DifferenceInDifferences, MultiPeriodDiD, TwoWayFixedEffects, and SyntheticDiD.
  • [Newly identified] P1: the implementation documents radial Bartlett Conley as “PSD-guaranteed” by citing Conley Eq. 3.14, but the source review says Conley’s explicit PSD Bartlett formula is the separable 2-D product window; the radial pairwise K(d/h) specialization is not the paper’s PSD formula.
  • Callable metric validation remains deferred and tracked in TODO.md, so it is informational only.
  • Tests were not run: numpy is missing and pytest is not installed in this environment.

Methodology

P1 — [Newly identified] Radial Bartlett is presented as Conley’s PSD-guaranteed Bartlett kernel

Locations: diff_diff/conley.py:L89-L92, docs/methodology/REGISTRY.md:L2958-L2960, diff_diff/guides/llms-full.txt:L1920-L1922, CHANGELOG.md:L11, docs/methodology/papers/conley-1999-review.md:L85-L93

Impact: The code implements K(u)=max(0,1-|u|) on scalar pairwise distance d_ij/h. The paper review states Conley’s explicit PSD Bartlett formula is the 2-D product lattice window, and that the 1-D pairwise Bartlett is a practitioner specialization not explicitly written in the paper. Because the registry/docstrings call the implemented kernel “PSD-guaranteed,” the code only warns for uniform; users can receive an indefinite Bartlett meat/vcov without the warning implied by the methodology source distinction.

Concrete fix: Either implement/document a true separable Conley Eq. 3.14 product kernel for the PSD-guaranteed path, or keep the current R-conleyreg-compatible radial Bartlett but add a REGISTRY.md **Note (deviation/source specialization):** explaining that it is the practitioner/R parity specialization, remove “PSD-guaranteed” language, and apply the negative-eigenvalue warning/guard to Bartlett as well when the computed meat/vcov is materially indefinite.

Code Quality

No additional unmitigated code-quality findings. The prior survey bypass is fixed at diff_diff/linalg.py:L2537-L2554, and covered by tests/test_conley_vcov.py:L662-L685.

Performance

No unmitigated findings. Dense O(n^2) Conley memory behavior is warned/documented, and the sparse fast path is tracked as deferred work.

Maintainability

P3 — Callable conley_metric output validation remains deferred and tracked

Locations: diff_diff/conley.py:L81-L82, TODO.md:L120

Impact: A malformed callable can return non-square, non-finite, negative, or asymmetric distances and fail opaquely or produce a nonsymmetric vcov.

Concrete fix: Already tracked in TODO.md; validate callable output shape, finiteness, non-negativity, and symmetry before forming the kernel.

Tech Debt

Tracked Conley deferrals for panel support, cluster/product kernels, survey/weighted Conley, SyntheticDiD support, and callable metric validation are acceptable as P3 informational items where the code fails fast.

Security

No security findings. Targeted secret-pattern scan over changed files found only benign references in .gitignore and changelog text.

Documentation/Tests

P3 — Minor stale Conley wording remains

Locations: diff_diff/conley.py:L128-L132, tests/test_conley_vcov.py:L687-L690

Impact: The missing-coordinates error still suggests TwoWayFixedEffects(conley_coords=...), even though TWFE rejects Conley in Phase 1. The test class docstring says panel estimators accept Conley and print a label, while the tests now assert rejection.

Concrete fix: Point the error message to LinearRegression / compute_robust_vcov, and update the test class docstring to describe fit-time panel rejection.

Test execution: Not run. python -c "import numpy" fails with ModuleNotFoundError: No module named 'numpy'; pytest --version fails with pytest: command not found.

Audit Confirmation

  • Sibling-surface mirror audit covered compute_robust_vcovsolve_olsLinearRegression, DiD ↔ MultiPeriodDiD ↔ TWFE, SyntheticDiD __init__set_params, results labels, guides, registry, TODO, and tests.
  • Pattern-wide grep found no new inline t_stat = ... / se anti-pattern in affected estimator/result paths.
  • Reciprocal guard check covered Conley+cluster, Conley+weights/survey, panel rejection, and SyntheticDiD constructor/setter rejection.
  • No changed GH Actions workflow path selector was in scope; transitive workflow dependency audit is not applicable.

Path to Approval

  1. Fix the Bartlett PSD contract: either implement a PSD-guaranteed separable Conley product kernel, or document the current radial Bartlett as a practitioner/R-parity specialization and remove the “PSD-guaranteed” claim.
  2. If keeping radial Bartlett, extend the negative-eigenvalue warning/guard beyond uniform so materially indefinite Bartlett meat/vcov does not pass under a PSD guarantee.
  3. Update the corresponding registry, conley.py docstring, guide/changelog wording, and add a regression test for the chosen PSD/warning contract.

igerber added a commit that referenced this pull request May 10, 2026
…h kernels

Address CI Codex review of PR #411 (P1 + P3):

P1 — The implementation called the radial 1-D Bartlett kernel
"PSD-guaranteed" by citing Conley 1999 Eq 3.14 + Andrews 1991. Conley's
explicit PSD Bartlett formula (Eq 3.14, page 12) is the 2-D separable
product window `(1 - |j|/L_M)(1 - |k|/L_N)` indexed on a lattice; the
1-D radial form on pairwise distance that diff-diff and R `conleyreg`
implement is a practitioner specialization (Hsiang 2010, Colella et
al. 2019) that is not explicitly written in the paper and is therefore
not formally PSD-guaranteed. Reframe the kernel docstring around the
practitioner-specialization framing, drop the PSD-guaranteed claim, and
lift the meat-eigenvalue PSD guard out of the uniform-only branch so
it fires for both supported kernels. The warning message now names
the active kernel and explicitly states neither radial form is
formally PSD. New regression test
test_indefinite_meat_warning_fires_for_bartlett locks the lifted guard
by patching `_bartlett_kernel` to return an aggressively indefinite
matrix and asserting the warning surfaces with the kernel name.

P3 — Stale wording cleanup:
- conley.py:128-132 missing-coords error message pointed at
  TwoWayFixedEffects(conley_coords=...) even though TWFE rejects
  Conley in Phase 1; redirect to LinearRegression / compute_robust_vcov.
- TestConleyEstimatorIntegration class docstring claimed panel
  estimators accept Conley and print a label; rewrote to describe
  fit-time panel rejection.

Doc surfaces (REGISTRY ConleySpatialHAC kernel section, llms-full.txt
kernels block, CHANGELOG `conley_kernel` description, conley-1999-
review.md PSD-failure note) updated to reflect the both-kernels guard
and the radial-specialization framing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 9b012086c73e64780b9c3fcefd3d61b9591469d5


PR Review: Conley Spatial HAC Re-review

Overall assessment: ✅ Looks good

Executive Summary

  • No unmitigated P0/P1 findings found in this re-review.
  • Prior Bartlett PSD methodology issue is resolved: radial Bartlett is now documented as a practitioner/R-parity specialization, and both Bartlett and uniform kernels use the negative-eigenvalue warning path.
  • Panel estimator behavior is now consistent: DifferenceInDifferences, MultiPeriodDiD, and TwoWayFixedEffects reject vcov_type="conley" at fit-time pending Phase 2 space-time HAC support.
  • Unsupported combinations (cluster_ids, weights, survey design, SyntheticDiD) fail fast or are tracked in TODO.md.
  • Tests were not run because this environment lacks pytest and numpy.

Methodology

P3 — Informational: radial Bartlett is a documented source specialization

Impact: The implementation uses the radial K(d_ij/h) Bartlett form matching R conleyreg, not Conley’s explicitly PSD-guaranteed 2-D separable product window. This is now documented in the registry and code, so it is not an unmitigated methodology defect.

Concrete fix: No action required. The registry note and runtime warning contract are present at docs/methodology/REGISTRY.md:L2958-L2963 and diff_diff/conley.py:L88-L100, with the warning check at diff_diff/conley.py:L260-L278.

Code Quality

No unmitigated findings. Conley dispatch is centralized through compute_robust_vcov / _compute_robust_vcov_numpy, and unsupported cluster_ids / weights fail fast at diff_diff/linalg.py:L1080-L1097.

Performance

No unmitigated findings. Dense O(n^2) behavior is warned in code at diff_diff/conley.py:L184-L192, and the sparse fast path is tracked as deferred work.

Maintainability

No unmitigated findings. The sibling surfaces are mirrored: compute_robust_vcov, solve_ols, LinearRegression, DiD/MultiPeriod/TWFE rejection paths, and SyntheticDiD.__init__ / set_params.

Tech Debt

P3 — Callable conley_metric output validation is tracked

Impact: A malformed callable can return a non-square, asymmetric, negative, or non-finite distance matrix, producing opaque errors or nonsymmetric covariance output.

Concrete fix: Already tracked in TODO.md:L120; validate callable output shape, finiteness, non-negativity, and symmetry before forming the kernel.

Security

No findings. Targeted secret-pattern scan over changed files found only benign references in .gitignore and changelog text.

Documentation/Tests

P3 — Minor docstring parameter list drift

Impact: compute_robust_vcov’s top-level behavior list includes Conley, but the vcov_type parameter line still lists only "classical", "hc1", "hc2", and "hc2_bm", and the conley_* keyword parameters are not separately documented in the parameter block. This is documentation-only; runtime dispatch is wired.

Concrete fix: Update diff_diff/linalg.py:L1220-L1226 to include "conley" and add parameter entries for conley_coords, conley_cutoff_km, conley_metric, and conley_kernel.

Test execution: Not run. pytest is not installed, and python imports fail with ModuleNotFoundError: No module named 'numpy'.

Audit Confirmation

  • Sibling-surface mirror audit covered compute_robust_vcovsolve_olsLinearRegression, DiD ↔ MultiPeriodDiD ↔ TWFE, and SyntheticDiD.__init__set_params.
  • Pattern-wide grep found no new Conley inline inference anti-pattern; Conley paths use safe_inference through existing result flows.
  • Reciprocal guard check covered Conley+cluster, Conley+weights, Conley+survey, panel rejection, and SyntheticDiD constructor/setter rejection.
  • No GH Actions workflow paths: changes were in this diff, so transitive workflow dependency audit is not applicable.

igerber added a commit that referenced this pull request May 10, 2026
Address P3 doc-drift finding from CI Codex review of PR #411 R-final.
The function-level docstring already lists Conley in the dispatch
description, but the Parameters section's `vcov_type` enum line and
the four `conley_*` keyword-only kwargs were not separately documented.
Added `"conley"` to the enum and four new parameter entries
(``conley_coords``, ``conley_cutoff_km``, ``conley_metric``,
``conley_kernel``) describing the cross-sectional-only Phase 1 contract
and the both-kernel indefiniteness warning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: e1d36f5ca6a6f4f0ffc2ffb0d915416c019e5045


PR Review: Conley Spatial HAC Re-review

Overall assessment: ✅ Looks good

Executive Summary

  • No unmitigated P0/P1 findings found in this re-review.
  • Conley methodology is now aligned with docs/methodology/REGISTRY.md: Phase 1 is cross-sectional only, panel estimators reject, no default bandwidth is introduced, and radial Bartlett is documented as an R-parity practitioner specialization.
  • Unsupported combinations fail fast: Conley + cluster/weights/survey, panel estimators, and SyntheticDiD all reject instead of silently computing a mismatched variance.
  • Remaining issues are P3 documentation/tech-debt cleanup only and do not block approval.
  • Tests were attempted but could not run because pytest is not installed in this environment.

Methodology

P3 — Informational: radial Bartlett is a documented source specialization

Impact: The implementation uses radial K(d_ij / h) Bartlett matching R conleyreg, not Conley’s explicitly PSD-guaranteed 2-D separable lattice window. This is documented in the registry and code, and both Bartlett/uniform paths warn on materially negative meat eigenvalues, so this is not an unmitigated methodology defect.

Concrete fix: No action required. See docs/methodology/REGISTRY.md:L2958-L2963 and diff_diff/conley.py:L88-L100, with the warning path at diff_diff/conley.py:L260-L278.

Code Quality

No unmitigated findings. Conley dispatch is centralized through compute_robust_vcov / _compute_robust_vcov_numpy, with explicit guards for unsupported cluster_ids and weights at diff_diff/linalg.py:L1080-L1097.

Performance

No unmitigated findings. Dense O(n^2) memory behavior is warned at validation time for large n in diff_diff/conley.py:L184-L192; sparse acceleration is deferred and tracked.

Maintainability

P3 — Dead Conley TWFE forwarding remains behind an unconditional rejection

Impact: TwoWayFixedEffects.fit() rejects vcov_type="conley" before fitting at diff_diff/twfe.py:L146-L163, but later code still materializes/forwards Conley coordinates at diff_diff/twfe.py:L328-L370. This is unreachable today and not a correctness issue, but it can confuse future maintenance.

Concrete fix: Either remove the unreachable block now or mark it explicitly as Phase 2 scaffolding.

Tech Debt

P3 — Callable conley_metric validation is tracked

Impact: A malformed callable metric can return a non-square, non-finite, negative, or asymmetric distance matrix, producing opaque errors or nonsymmetric covariance output.

Concrete fix: Already tracked in TODO.md:L120; add shape, finiteness, non-negativity, and symmetry validation before kernel formation.

Security

No findings. Targeted secret-pattern scan over changed files found only benign references in .gitignore and changelog text.

Documentation/Tests

P3 — Stale Conley paper-review notes conflict with the registry

Impact: The authoritative registry and code now correctly document radial Bartlett as not formally PSD-guaranteed, but docs/methodology/papers/conley-1999-review.md:L213-L220 and docs/methodology/papers/conley-1999-review.md:L240-L246 still say Bartlett is PSD by construction and only uniform needs the negative-eigenvalue warning. This is documentation drift, not a runtime defect.

Concrete fix: Update the paper-review checklist/table to distinguish Conley’s separable lattice Bartlett from the radial pairwise Bartlett implemented here, and mention that both kernels use the warning path.

Test execution: python -m pytest tests/test_conley_vcov.py -q was attempted and failed with /usr/bin/python: No module named pytest.

Audit Confirmation

  • Sibling-surface mirror audit covered compute_robust_vcov_compute_robust_vcov_numpysolve_olsLinearRegression, DiD ↔ MultiPeriodDiD ↔ TWFE, and SyntheticDiD.__init__set_params.
  • Pattern-wide grep covered inline inference / se > 0 else 0.0 patterns and Conley guard propagation; no new Conley inline inference issue found.
  • Reciprocal checks covered Conley + cluster, weights, survey, panel estimators, and SyntheticDiD constructor/setter rejection paths.
  • No GH Actions workflow paths: changes were in the provided changed-file list, so the transitive workflow dependency audit is not applicable.

igerber added a commit that referenced this pull request May 10, 2026
Address two P3 findings from CI Codex review of PR #411:

P3 (Maintainability) — DiD/MultiPeriodDiD/TWFE all reject vcov_type="conley"
unconditionally at fit-time, but each fit() still materialized
`_conley_coords_array` from data and forwarded `conley_coords`,
`conley_cutoff_km`, `conley_metric`, `conley_kernel` to LinearRegression /
solve_ols. Those code paths were unreachable behind the unconditional
NotImplementedError raise. Removed the dead extraction + arg-passes from
all three estimators. The constructor still accepts the conley_* kwargs
for sklearn-style API symmetry (set_params/get_params round-trip works);
they have no effect on the panel paths.

P3 (Documentation) — `docs/methodology/papers/conley-1999-review.md`
Requirements checklist and Tuning Parameters table still said the
Bartlett kernel is "PSD by construction" and only flagged uniform as
needing the negative-eigenvalue warning. Updated both surfaces to spell
out the radial 1-D pairwise specialization vs Conley's explicit 2-D
separable PSD lattice formula (Eq 3.14) and to apply the warning to
both kernels — matching the registry and the runtime contract.

271 targeted regression tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: af246a17a55c7ab35874a6f7b5a75e9e19d7a6be


PR Review: Conley Spatial HAC Re-review

Overall Assessment

Looks good — No unmitigated P0/P1 findings found in this re-review.

Executive Summary

  • The prior methodology risk is addressed: DifferenceInDifferences, MultiPeriodDiD, and TwoWayFixedEffects now reject vcov_type="conley" at fit-time instead of applying cross-sectional Conley to panel rows.
  • The registry documents Conley Phase 1 as cross-sectional only, with no default bandwidth and with the radial Bartlett implementation explicitly labeled as a source specialization/deviation.
  • Unsupported combinations fail fast: Conley + cluster/weights/survey, panel estimators, and SyntheticDiD.
  • Remaining issues are P3 documentation/maintenance/tracked-debt items only.
  • I could not run tests in this environment because pytest and numpy are not installed.

Methodology

P3 — Informational: radial Bartlett is a documented source specialization

Impact: The implementation uses K(d_ij / h) radial Bartlett, matching R conleyreg, rather than Conley 1999’s explicitly PSD-guaranteed 2-D separable lattice Bartlett. This is documented in the registry and implementation, and the implementation warns on materially negative meat eigenvalues for both Bartlett and uniform kernels, so this is not an unmitigated methodology defect.

Concrete fix: No action required. See docs/methodology/REGISTRY.md:L2958-L2963 and diff_diff/conley.py:L88-L100, with the eigenvalue warning at diff_diff/conley.py:L260-L278.

Code Quality

No unmitigated findings.

The Conley dispatch and guards are centralized through compute_robust_vcov / _compute_robust_vcov_numpy, with explicit conley + cluster_ids and conley + weights rejection at diff_diff/linalg.py:L1080-L1097. LinearRegression also front-door rejects vcov_type="conley" with survey variance at diff_diff/linalg.py:L2560-L2577.

Performance

P3 — Informational: dense O(n²) Conley path is warned and tracked

Impact: Phase 1 builds a dense pairwise distance matrix, which is expected but can be expensive for large n.

Concrete fix: No blocking action required. The implementation emits a warning for n > 20_000 at diff_diff/conley.py:L184-L192, and the sparse fast path is tracked in TODO.md:L116.

Maintainability

P3 — Unreachable TWFE Conley auto-cluster branch remains

Impact: TwoWayFixedEffects.fit() rejects vcov_type="conley" before fitting at diff_diff/twfe.py:L146-L163, but a later elif self.vcov_type == "conley" branch still disables auto-clustering at diff_diff/twfe.py:L234-L240. This is unreachable today and not a correctness issue, but it can confuse future Phase 2 work.

Concrete fix: Remove the unreachable block or mark it explicitly as Phase 2 scaffolding.

Tech Debt

P3 — Callable conley_metric validation is tracked

Impact: A malformed callable metric can return a non-square, non-finite, negative, or asymmetric matrix, producing opaque errors or nonsymmetric covariance output.

Concrete fix: Already tracked in TODO.md:L120; add shape, finiteness, non-negativity, and symmetry validation before kernel formation.

Security

No findings. Targeted secret-pattern scan over changed Conley benchmark/test/source files found no hits.

Documentation/Tests

P3 — resolve_vcov_type docstring omits conley

Impact: _VALID_VCOV_TYPES includes "conley" at diff_diff/linalg.py:L1003, but the resolve_vcov_type docstring still lists only {"classical", "hc1", "hc2", "hc2_bm"} and omits "conley" in the return set at diff_diff/linalg.py:L1115-L1130. Runtime behavior is correct; this is documentation drift.

Concrete fix: Update the resolve_vcov_type docstring to include "conley" in the allowed and returned values.

Test execution: python -m pytest tests/test_conley_vcov.py -q failed with /usr/bin/python: No module named pytest. A Python import smoke test also failed with ModuleNotFoundError: No module named 'numpy'.

Audit Confirmation

  • Sibling-surface mirror audit covered compute_robust_vcov_compute_robust_vcov_numpysolve_olsLinearRegression, DiD ↔ MultiPeriodDiD ↔ TWFE, and SyntheticDiD.__init__set_params.
  • Pattern-wide grep covered inline inference / se > 0 else 0.0 patterns and Conley guard propagation; no new Conley inference anti-pattern found.
  • Reciprocal checks covered Conley + cluster, weights, survey, panel estimators, and SyntheticDiD constructor/setter rejection paths.
  • GH Actions workflow paths: audit is not applicable because no workflow files are changed.

igerber added a commit that referenced this pull request May 10, 2026
…esolve_vcov_type docstring

Address two P3 documentation/maintenance findings from CI Codex review of
PR #411 R-final:

- twfe.py:234-240: an `elif self.vcov_type == "conley"` branch in the
  cluster-resolution chain disabled the auto-cluster default for Conley.
  Unreachable behind the unconditional Conley reject at twfe.py:146-163.
  Removed.
- linalg.py::resolve_vcov_type docstring: the "Rules" section and
  Returns docstring listed only `{"classical", "hc1", "hc2", "hc2_bm"}`,
  omitting `"conley"` even though `_VALID_VCOV_TYPES` includes it and
  the runtime path already accepts it. Updated both surfaces; the source
  of truth pointer (`_VALID_VCOV_TYPES`) is now named explicitly so
  future enum additions don't need a docstring sweep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 10, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 84d7572416933ec8a92b1349995e4975fc84157e


PR Review: Conley Spatial HAC Re-review

Overall Assessment

Looks good — No unmitigated P0/P1 findings found.

Executive Summary

  • The Conley Phase 1 scope is now consistent: cross-sectional compute_robust_vcov / LinearRegression are supported, while DiD, MultiPeriodDiD, and TWFE reject panel Conley at fit-time.
  • The radial Bartlett implementation is documented as a source specialization/deviation from Conley’s PSD lattice Bartlett, so it is not a methodology defect.
  • Unsupported combinations fail fast: Conley + cluster/weights/survey, panel estimators, and SyntheticDiD.
  • Pattern-wide inference grep found no new inline t_stat = effect / se or partial-NaN inference anti-pattern in the affected Conley path.
  • I could not run tests because this environment lacks pytest and numpy.

Methodology

P3 — Informational: radial Bartlett is documented as a source specialization

Impact: The implementation uses radial K(d_ij / h) Bartlett matching R conleyreg, not Conley 1999’s explicitly PSD-guaranteed 2-D separable lattice Bartlett. This is documented in the registry and code, and the meat eigenvalue warning applies to both Bartlett and uniform kernels.

Concrete fix: No action required.

Locations: docs/methodology/REGISTRY.md:L2958-L2962, diff_diff/conley.py:L88-L100, diff_diff/conley.py:L260-L278.

Code Quality

No unmitigated findings. The Conley dispatch and validation are centralized through _validate_vcov_args, compute_robust_vcov, _compute_robust_vcov_numpy, solve_ols, and LinearRegression.fit.

Locations: diff_diff/linalg.py:L1080-L1098, diff_diff/linalg.py:L1663-L1679, diff_diff/linalg.py:L2561-L2578.

Performance

P3 — Informational: dense O(n²) Conley path is warned and tracked

Impact: Phase 1 materializes a dense pairwise distance matrix, which can be expensive for large cross sections.

Concrete fix: No blocking action required. The implementation warns at large n, and the sparse k-d-tree follow-up is tracked.

Locations: diff_diff/conley.py:L184-L192, TODO.md:L116.

Maintainability

P3 — Minor: MultiPeriodDiD has a redundant Conley + absorb pre-guard

Impact: MultiPeriodDiD.fit() first raises for absorb and vcov_type == "conley", then immediately has a general Conley panel rejection. The special message suggests dropping absorb=, but dropping it still raises because Phase 1 rejects all MultiPeriodDiD Conley fits.

Concrete fix: Remove the absorb-specific Conley block or change its message to point to the unconditional Phase 2 panel rejection.

Location: diff_diff/estimators.py:L1377-L1397.

Tech Debt

P3 — Tracked: callable conley_metric validation is deferred

Impact: A malformed callable metric can return invalid distances and produce opaque errors or nonsymmetric output. This is not a blocker because it is documented as deferred technical debt.

Concrete fix: Already tracked; add shape, finiteness, non-negativity, and symmetry validation before kernel formation.

Locations: diff_diff/conley.py:L66-L82, TODO.md:L120.

Security

No findings. Targeted secret-pattern scan over the changed Conley source, docs, tests, and benchmark files returned no hits.

Documentation/Tests

P3 — Minor docs drift: Colella paper review still says TWFE Conley is a Phase 1 parity target

Impact: The registry and code now correctly make panel Conley Phase 2, but the Colella review still says “Phase 1 parity target” for TWFE + Conley. The registry is authoritative, so this is not a methodology blocker, but it may mislead future work.

Concrete fix: Update that line to say TWFE/acreg parity belongs to Phase 2 space-time Conley.

Location: docs/methodology/papers/colella-et-al-2019-review.md:L193.

Test execution: python -m pytest tests/test_conley_vcov.py -q failed with /usr/bin/python: No module named pytest. Import smoke also failed with ModuleNotFoundError: No module named 'numpy'.

Audit Confirmation

  • Sibling-surface mirror audit covered compute_robust_vcov_compute_robust_vcov_numpysolve_olsLinearRegression, DiD ↔ MultiPeriodDiD ↔ TWFE, and SyntheticDiD.__init__get_params / set_params.
  • Pattern-wide grep covered inline inference / se > 0 else 0.0, Conley guards, and Conley kwarg propagation.
  • Reciprocal checks covered Conley + cluster, weights, survey, panel estimators, SyntheticDiD constructor/setter rejection, and direct helper vs public dispatch.
  • Workflow dependency audit: no workflow files are in the changed-file list, so transitive workflow path review is not applicable.

igerber and others added 7 commits May 10, 2026 20:57
Wires Conley spatial-HAC standard errors into DifferenceInDifferences,
TwoWayFixedEffects, and MultiPeriodDiD via four new keyword-only kwargs:
conley_coords (lat/lon column tuple), conley_cutoff_km (required, no
default per no-silent-failures), conley_metric ("haversine" default,
"euclidean", or callable), conley_kernel ("bartlett" default, "uniform").

Variance estimator (Conley 1999 Eq 4.2): Var̂(β) = (X'X)^{-1} ·
(Σ_{i,j} K(d_ij/h) X_i ε_i ε_j X_j') · (X'X)^{-1}. FWL composes cleanly
because the meat depends only on scores X·ε which within-transformation
preserves -- TwoWayFixedEffects with conley is supported, unlike hc2/hc2_bm
which need the full hat matrix. TWFE auto-cluster-at-unit is disabled when
vcov_type="conley"; explicit cluster= raises NotImplementedError (combined
product kernel deferred to Phase 2).

Helpers live in new module diff_diff/conley.py (separated from linalg.py
to keep the linear-algebra backend focused; revisited from the original
plan when Phase 2's k-d-tree fast path was scoped).

Cross-language parity vs R conleyreg (Düsterhöft 2021, CRAN v0.1.9) at
≤1e-6 on three benchmark fixtures; observed max abs diff 5.7e-16.
Earth radius 6371.01 km matches conleyreg::haversine_dist exactly.
Generation: cd benchmarks/R && Rscript generate_conley_golden.R.

SyntheticDiD(vcov_type="conley") raises TypeError (uses bootstrap variance,
not analytical sandwich; tracked in TODO.md). conley + cluster_ids /
weights / survey_design / absorb each raise NotImplementedError with
references to the deferral phase.

Other changes: .gitignore /papers/ rule anchored to repo root so
docs/methodology/papers/ is no longer silently swallowed; 5 paper-review
markdowns committed (Conley 1999, Colella 2019, Clarke 2017, Butts 2023,
Butts 2021) as the methodology basis for REGISTRY ConleySpatialHAC and
the Phase 3 ring-indicator rescope flagged in BRIEFING.md.

70 new Conley tests pass (67 internal + 3 R parity); 442 tests pass on
the targeted regression surface (test_linalg, test_linalg_hc2_bm,
test_estimators_vcov_type, test_estimators, test_methodology_twfe).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes two no-silent-failure gaps surfaced after the initial Phase 1 commit:

1. SyntheticDiD.set_params() now mirrors __init__'s Conley rejection contract.
   The constructor correctly raises TypeError on vcov_type="conley" /
   conley_*=... but set_params() previously only checked hasattr(self, key)
   and silently accepted those kwargs because SyntheticDiD inherits the
   conley_* attributes from DifferenceInDifferences. A user calling
   est.set_params(vcov_type="conley", conley_cutoff_km=100.0) followed by
   est.fit(...) would have gotten the bootstrap/jackknife/placebo variance
   silently with no Conley computation -- forbidden by feedback_no_silent_failures.

   The new set_params() rejects non-None vcov_type and any non-None conley_*
   kwarg with TypeError before mutation; None values for these keys are
   passthrough no-ops so get_params() -> set_params() round-trips cleanly.
   SyntheticDiD.get_params() now surfaces the inherited conley_* keys with
   None values for sklearn-style API consistency.

2. TwoWayFixedEffects(vcov_type="conley", inference="wild_bootstrap") now
   raises NotImplementedError. Conley analytical spatial-HAC and wild cluster
   bootstrap are different inference paths; combining them would route the
   bootstrap branch with cluster_ids=None (TWFE auto-cluster is disabled
   under Conley) and fail with a non-targeted error inside wild_bootstrap_se.
   Use inference='analytical' for Conley spatial HAC, or vcov_type='hc1'
   with inference='wild_bootstrap'.

3. DifferenceInDifferences and MultiPeriodDiD class docstrings now list
   vcov_type="conley" in the enum and document the four conley_* params
   (previously the Conley path was documented in REGISTRY/CHANGELOG/llms.txt
   but the in-code class docs still listed only {classical, hc1, hc2, hc2_bm}).

4. New tests:
   - SyntheticDiD().set_params(vcov_type="conley") raises TypeError + state
     unchanged
   - SyntheticDiD().set_params(conley_cutoff_km=100.0) raises + state unchanged
   - SyntheticDiD().get_params() includes conley_* keys with None values;
     round-trip set_params(**get_params()) is a no-op
   - TwoWayFixedEffects(vcov_type="conley", inference="wild_bootstrap") raises

5. TODO.md entries added for deferred follow-ups: callable-conley_metric
   shape/finiteness/symmetry validation, common Conley estimator-validator
   helper extraction, and a stronger TWFE Conley FWL invariance test that
   actually compares TWFE-within Conley to a full-dummy FE design (the
   current test only asserts finite SEs).

74 Conley tests pass (70 prior + 4 new); 261 tests pass on the targeted
regression surface (test_conley_vcov, test_estimators_vcov_type, test_linalg,
test_linalg_hc2_bm).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ring

Address CI Codex review of PR #411 (P1#1 + P1#2):

P1#1 — Panel estimators with vcov_type="conley" silently produced wrong
SE because cross-sectional Conley over (unit, time) rows treated same-
unit cross-time pairs as d_ij=0 -> K=1, mishandling the space-time HAC.
Phase 1 supports cross-sectional Conley only; reject panel fits at
fit-time on DifferenceInDifferences, TwoWayFixedEffects, and
MultiPeriodDiD with NotImplementedError. Practitioners pre-collapse to
per-unit first-differences and call compute_robust_vcov directly. Phase
2 will add the space-time product kernel (Driscoll-Kraay) and lift the
rejection. Granular Conley-arg validation collapsed into the single
unconditional reject (cluster/absorb/coords/cutoff combinations all
hit the same path).

P1#2 — conley_metric was dropped at the result boundary and
_format_vcov_label hard-coded "km" for the cutoff label even when
metric was "euclidean". With panels rejected, the conley_cutoff_km /
conley_kernel fields on DiDResults / MultiPeriodDiDResults are now
unreachable; remove the dead fields, the dead arg passes from
estimators.py / twfe.py, and the dead "conley" branch in
_format_vcov_label.

Tests added: TWFE / DiD / MPD panel-rejection regressions, including a
repeated-coords-across-periods regression per the CI reviewer's
recommendation. 70 Conley tests + 401 targeted regression tests pass.

REGISTRY / CHANGELOG / llms.txt / README / TODO updated to reflect
that the only supported Phase 1 Conley path is direct
LinearRegression / compute_robust_vcov on a single-period design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address CI Codex review of PR #411 (P1 + P2):

P1 — LinearRegression(vcov_type="conley", survey_design=...) silently
bypassed the documented Conley+survey rejection. The downstream
_validate_vcov_args (inside compute_robust_vcov) rejects this combination,
but LinearRegression.fit() sets return_vcov=False on the solve_ols call
when survey vcov is needed (so the linalg validator never runs), and the
survey vcov path then overwrites vcov_ with a non-Conley variance under a
Conley request. Front-door the rejection at LinearRegression entry so the
contract is enforced uniformly. New regression test
test_linear_regression_conley_with_survey_design_raises locks the new
guard with a make_pweight_design fixture.

P2 — DifferenceInDifferences and MultiPeriodDiD docstrings still listed
"conley" as a vcov_type option and described the conley_* kwargs as if
the path were reachable. Updated both to spell out the Phase 1 panel
rejection and point users at compute_robust_vcov / LinearRegression for
cross-sectional Conley. Rewrote the llms-full.txt Conley section around
the LinearRegression / compute_robust_vcov surface, replaced the panel-
estimator example with a cross-sectional one, and listed the panel
rejection in the restrictions table.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…h kernels

Address CI Codex review of PR #411 (P1 + P3):

P1 — The implementation called the radial 1-D Bartlett kernel
"PSD-guaranteed" by citing Conley 1999 Eq 3.14 + Andrews 1991. Conley's
explicit PSD Bartlett formula (Eq 3.14, page 12) is the 2-D separable
product window `(1 - |j|/L_M)(1 - |k|/L_N)` indexed on a lattice; the
1-D radial form on pairwise distance that diff-diff and R `conleyreg`
implement is a practitioner specialization (Hsiang 2010, Colella et
al. 2019) that is not explicitly written in the paper and is therefore
not formally PSD-guaranteed. Reframe the kernel docstring around the
practitioner-specialization framing, drop the PSD-guaranteed claim, and
lift the meat-eigenvalue PSD guard out of the uniform-only branch so
it fires for both supported kernels. The warning message now names
the active kernel and explicitly states neither radial form is
formally PSD. New regression test
test_indefinite_meat_warning_fires_for_bartlett locks the lifted guard
by patching `_bartlett_kernel` to return an aggressively indefinite
matrix and asserting the warning surfaces with the kernel name.

P3 — Stale wording cleanup:
- conley.py:128-132 missing-coords error message pointed at
  TwoWayFixedEffects(conley_coords=...) even though TWFE rejects
  Conley in Phase 1; redirect to LinearRegression / compute_robust_vcov.
- TestConleyEstimatorIntegration class docstring claimed panel
  estimators accept Conley and print a label; rewrote to describe
  fit-time panel rejection.

Doc surfaces (REGISTRY ConleySpatialHAC kernel section, llms-full.txt
kernels block, CHANGELOG `conley_kernel` description, conley-1999-
review.md PSD-failure note) updated to reflect the both-kernels guard
and the radial-specialization framing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address P3 doc-drift finding from CI Codex review of PR #411 R-final.
The function-level docstring already lists Conley in the dispatch
description, but the Parameters section's `vcov_type` enum line and
the four `conley_*` keyword-only kwargs were not separately documented.
Added `"conley"` to the enum and four new parameter entries
(``conley_coords``, ``conley_cutoff_km``, ``conley_metric``,
``conley_kernel``) describing the cross-sectional-only Phase 1 contract
and the both-kernel indefiniteness warning.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address two P3 findings from CI Codex review of PR #411:

P3 (Maintainability) — DiD/MultiPeriodDiD/TWFE all reject vcov_type="conley"
unconditionally at fit-time, but each fit() still materialized
`_conley_coords_array` from data and forwarded `conley_coords`,
`conley_cutoff_km`, `conley_metric`, `conley_kernel` to LinearRegression /
solve_ols. Those code paths were unreachable behind the unconditional
NotImplementedError raise. Removed the dead extraction + arg-passes from
all three estimators. The constructor still accepts the conley_* kwargs
for sklearn-style API symmetry (set_params/get_params round-trip works);
they have no effect on the panel paths.

P3 (Documentation) — `docs/methodology/papers/conley-1999-review.md`
Requirements checklist and Tuning Parameters table still said the
Bartlett kernel is "PSD by construction" and only flagged uniform as
needing the negative-eigenvalue warning. Updated both surfaces to spell
out the radial 1-D pairwise specialization vs Conley's explicit 2-D
separable PSD lattice formula (Eq 3.14) and to apply the warning to
both kernels — matching the registry and the runtime contract.

271 targeted regression tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…esolve_vcov_type docstring

Address two P3 documentation/maintenance findings from CI Codex review of
PR #411 R-final:

- twfe.py:234-240: an `elif self.vcov_type == "conley"` branch in the
  cluster-resolution chain disabled the auto-cluster default for Conley.
  Unreachable behind the unconditional Conley reject at twfe.py:146-163.
  Removed.
- linalg.py::resolve_vcov_type docstring: the "Rules" section and
  Returns docstring listed only `{"classical", "hc1", "hc2", "hc2_bm"}`,
  omitting `"conley"` even though `_VALID_VCOV_TYPES` includes it and
  the runtime path already accepts it. Updated both surfaces; the source
  of truth pointer (`_VALID_VCOV_TYPES`) is now named explicitly so
  future enum additions don't need a docstring sweep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igerber igerber force-pushed the spillover-conley branch from 84d7572 to 5806b33 Compare May 11, 2026 00:57
@igerber
Copy link
Copy Markdown
Owner Author

igerber commented May 11, 2026

/ai-review

@github-actions
Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 5806b33a7424d7dda4aab48632bedee2f6306563


PR Review: Conley Spatial HAC SE Re-review

Overall Assessment

⚠️ Needs changes — No P0/P1 methodology or correctness blockers found in the loaded source, but there are unmitigated P2 claim/documentation drift issues that block ✅ under the review criteria.

Executive Summary

  • Methodology implementation now aligns with the provided REGISTRY.md: Conley is supported only for cross-sectional compute_robust_vcov / LinearRegression, while DiD, MultiPeriodDiD, and TWFE reject panel Conley at fit-time.
  • The Conley sandwich implementation matches the documented pairwise-score formula and includes required guards for missing bandwidth, invalid coordinates, unsupported cluster/weight combinations, and dense O(n²) memory.
  • Prior P1-style concern about silently applying cross-sectional Conley to panel estimators appears resolved in the visible code and tests.
  • Remaining blockers are P2 documentation/claim drift: the PR body still advertises panel/TWFE support, and several public docstrings do not document newly added Conley parameters or rejection behavior.
  • Existing P3 items remain: callable metric validation is tracked in TODO.md, dense O(n²) path is warned/tracked, MultiPeriodDiD has a redundant/misleading Conley+absorb pre-guard, and the Colella paper review still describes TWFE Conley as a Phase 1 target.

Methodology

P3 — Informational: radial Bartlett specialization is documented as a deviation/source specialization

Impact: The implementation uses radial 1-D Bartlett on pairwise distance, matching R conleyreg, rather than Conley (1999)’s explicitly PSD-guaranteed 2-D separable lattice Bartlett. This is a methodology deviation from the paper’s formal PSD construction, but it is explicitly documented with a **Note (deviation / source specialization):** in REGISTRY.md, and the code warns on materially negative meat eigenvalues for both kernels. Per review policy, this is not a defect.

Concrete fix: No action required.

Locations:

  • docs/methodology/REGISTRY.md## ConleySpatialHAC, “Note (deviation / source specialization)”
  • diff_diff/conley.py::_bartlett_kernel
  • diff_diff/conley.py::_compute_conley_vcov

P3 — Informational: panel Conley is correctly rejected per current registry scope

Impact: The changed registry and changelog state Phase 1 supports cross-sectional Conley only and rejects DifferenceInDifferences, MultiPeriodDiD, and TwoWayFixedEffects at fit-time. The visible implementation follows that contract:

  • DifferenceInDifferences.fit() raises for vcov_type == "conley".
  • MultiPeriodDiD.fit() raises for vcov_type == "conley".
  • TwoWayFixedEffects.fit() raises for vcov_type == "conley".

This resolves the main methodology risk of silently applying cross-sectional Conley over panel (unit, time) rows.

Concrete fix: No action required.

Locations:

  • diff_diff/estimators.py::DifferenceInDifferences.fit
  • diff_diff/estimators.py::MultiPeriodDiD.fit
  • diff_diff/twfe.py::TwoWayFixedEffects.fit
  • docs/methodology/REGISTRY.md## ConleySpatialHAC

Code Quality

P3 — Minor: MultiPeriodDiD has a redundant and misleading Conley + absorb pre-guard

Impact: MultiPeriodDiD.fit() first raises a special NotImplementedError for absorb and vcov_type == "conley" suggesting “drop absorb= for cross-sectional Conley,” but the next unconditional guard rejects all MultiPeriodDiD(vcov_type="conley") fits anyway. The message can mislead users into thinking drop absorb= would make Conley available on MultiPeriodDiD.

Concrete fix: Remove the absorb and self.vcov_type == "conley" special case, or rewrite it to say MultiPeriodDiD rejects Conley regardless of absorb until the Phase 2 space-time HAC implementation.

Location: diff_diff/estimators.py::MultiPeriodDiD.fit

P3 — Tracked: callable conley_metric output validation is deferred

Impact: A user-supplied callable metric is accepted via np.asarray(metric(coords, coords)) without shape, finiteness, non-negativity, or symmetry validation. A malformed callable can produce opaque downstream errors or nonsymmetric variance output. This is tracked in TODO.md, so it is informational.

Concrete fix: In a follow-up, validate callable metric output before kernel formation:

  • shape exactly (n, n)
  • finite values
  • non-negative distances
  • diagonal near zero
  • symmetric within tolerance, or explicitly document/support directed metrics

Locations:

  • diff_diff/conley.py::_pairwise_distance_matrix
  • TODO.md — Conley callable metric validation row

Performance

P3 — Informational: dense O(n²) Conley implementation is warned and tracked

Impact: Phase 1 materializes a dense n × n distance matrix and kernel matrix. This can be memory-heavy for large cross sections. The implementation emits a warning for n > 20_000, and the sparse k-d-tree fast path is tracked in TODO.md.

Concrete fix: No blocking action required. Phase 2 should add the sparse neighbor-query path for compact-support kernels.

Locations:

  • diff_diff/conley.py::_validate_conley_kwargs
  • TODO.md — Conley space-time / sparse fast path row

Maintainability

P3 — Minor: test name/assertion implies radial Bartlett PSD despite registry caveat

Impact: tests/test_conley_vcov.py::TestConleyKernels::test_bartlett_psd_on_random_distances asserts a PSD property for a random radial Bartlett kernel matrix. The registry and code correctly state radial 1-D Bartlett is not formally PSD-guaranteed. Even if this seeded test passes today, the test name and invariant encode a stronger property than the methodology contract.

Concrete fix: Rename/reframe the test to avoid claiming guaranteed PSD, or replace it with a deterministic “code path produces finite kernel matrix” test. Keep the existing negative-eigenvalue warning test as the contract for indefinite cases.

Location: tests/test_conley_vcov.py::TestConleyKernels::test_bartlett_psd_on_random_distances


Tech Debt

P3 — Tracked: unsupported Conley combinations are explicitly deferred

Impact: The PR adds TODO rows for:

  • panel space-time Conley / Driscoll-Kraay product kernel,
  • Conley + cluster product kernel,
  • Conley + survey/weights,
  • SyntheticDiD + Conley,
  • callable metric validation.

These are visible limitations but are either fail-fast in code or tracked as follow-up work, so they do not block approval.

Concrete fix: No immediate action required beyond completing the tracked follow-ups.

Locations: TODO.md


Security

No security issues found in the loaded source. The added benchmark JSON appears to contain synthetic/generated numerical data, not secrets or PII.


Documentation/Tests

P2 — PR body still falsely advertises panel/TWFE Conley support

Impact: The visible registry, changelog, and implementation now say panel estimators reject vcov_type="conley" in Phase 1. However, the PR body still claims:

  • “Add vcov_type="conley" on DifferenceInDifferences, TwoWayFixedEffects, and MultiPeriodDiD
  • “FWL composes cleanly … TwoWayFixedEffects(vcov_type="conley", ...) is therefore supported”

This is a claim-vs-shipped mismatch. Users/reviewers reading the PR body could reasonably believe TWFE/DiD/MultiPeriodDiD Conley is available, when the shipped behavior is NotImplementedError.

Concrete fix: Update the PR body summary to match the shipped contract:

  • Conley is supported only on cross-sectional LinearRegression / compute_robust_vcov.
  • DiD, MultiPeriodDiD, and TWFE reject vcov_type="conley" at fit-time until Phase 2 space-time HAC.
  • Remove the statement that TWFE Conley is supported and that auto-cluster is disabled for TWFE Conley.

Location: PR body summary text

P2 — Public docstrings missing new Conley parameter documentation on cross-sectional public APIs

Impact: LinearRegression.__init__ adds public parameters conley_coords, conley_cutoff_km, conley_metric, and conley_kernel, and solve_ols() also accepts these parameters. compute_robust_vcov() documents them well, but the LinearRegression class docstring does not list the Conley parameters, and the solve_ols() docstring includes "conley" in the vcov_type set without documenting the conley_* parameters or the Conley dispatch details in its parameter list. This creates public-API docstring drift for the primary supported cross-sectional API.

Concrete fix: Add explicit parameter blocks to:

  1. diff_diff/linalg.py::LinearRegression class docstring:
    • conley_coords
    • conley_cutoff_km
    • conley_metric
    • conley_kernel
    • note unsupported cluster_ids / weights / survey_design combinations.
  2. diff_diff/linalg.py::solve_ols docstring:
    • add a "conley" bullet under vcov_type
    • document each conley_* parameter and required/unsupported combinations.

Locations:

  • diff_diff/linalg.py::LinearRegression docstring
  • diff_diff/linalg.py::solve_ols docstring

P2 — Public docstrings missing SyntheticDiD Conley rejection behavior

Impact: SyntheticDiD.__init__ now accepts vcov_type and conley_* kwargs only to reject them with TypeError, and REGISTRY.md / CHANGELOG.md explicitly claim SyntheticDiD(vcov_type="conley") raises. The public SyntheticDiD class docstring does not document these new rejected parameters or the reason for rejection. This is public-API docstring drift.

Concrete fix: Add a short parameter/notes block to SyntheticDiD documenting:

  • vcov_type and conley_* are not supported.
  • Passing vcov_type="conley" or any non-None conley_* kwarg raises TypeError.
  • Rationale: SyntheticDiD uses bootstrap/jackknife/placebo variance, not the analytical sandwich.

Location: diff_diff/synthetic_did.py::SyntheticDiD class docstring

P2 — Public TWFE docstring does not mention Conley rejection despite inherited constructor accepting the parameter

Impact: TwoWayFixedEffects inherits the DifferenceInDifferences constructor, so users can instantiate TwoWayFixedEffects(vcov_type="conley", conley_coords=..., ...). The fit-time rejection is correctly implemented, and the registry/changelog claim rejection. But the public TwoWayFixedEffects docstring does not document vcov_type="conley" rejection; it only discusses HC2/Bell-McCaffrey limitations.

Concrete fix: Add a vcov_type="conley" note to the TwoWayFixedEffects class docstring explaining Phase 1 rejection and the Phase 2 space-time HAC follow-up.

Location: diff_diff/twfe.py::TwoWayFixedEffects class docstring

P3 — Minor docs drift: Colella paper review still describes TWFE Conley as a Phase 1 parity target

Impact: The registry and implementation now make panel Conley a Phase 2 target, but the newly added Colella paper review contains language saying “Phase 1 parity target: vcov_method="conley" on TWFE must match acreg…”. This conflicts with the current authoritative registry. Because this is a paper-review note rather than the registry/API docs, classify as P3.

Concrete fix: Update the Colella review to say cross-sectional Conley is Phase 1; TWFE/acreg space-time Conley parity belongs to Phase 2.

Location: docs/methodology/papers/colella-et-al-2019-review.md — “Relation to Existing diff-diff Estimators” / “Phase 1 parity target” paragraph


Path to Approval

  1. Fix the PR body claim mismatch: revise the PR description so it no longer claims DiD/TWFE/MultiPeriodDiD Conley support in Phase 1, and instead states those estimators reject until Phase 2 space-time HAC.
  2. Update public docstrings for supported cross-sectional APIs:
    • Add conley_coords, conley_cutoff_km, conley_metric, conley_kernel parameter documentation to LinearRegression.
    • Add the same parameter documentation and a vcov_type="conley" bullet to solve_ols().
  3. Update public docstrings for rejected public surfaces:
    • Document SyntheticDiD’s TypeError rejection for vcov_type / conley_*.
    • Document TwoWayFixedEffects(vcov_type="conley") fit-time rejection in the class docstring.
  4. Optional but recommended P3 cleanup:
    • Remove or rewrite the redundant MultiPeriodDiD(absorb=..., vcov_type="conley") special guard.
    • Update the Colella paper review’s TWFE Phase 1 wording.
    • Rename/reframe the Bartlett PSD test to avoid encoding a stronger-than-documented PSD guarantee.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant