docs(hhtl): GridLake + PR-X14′ substrate-pair pre-sprint prompts#163
Merged
Conversation
…rate identity made concrete Folds the gaussian-splat session's feedback into a copy-paste-ready pre-sprint kickoff for PR-X1 (MultiLaneColumn carrier) + PR-X2 (#[derive(SoA)] proc-macro). Grounded verbatim in PR #162's column-substrate identity so nothing floats — every claim cites a line range in a doc that just merged to master. ## Quoted verbatim from PR #162 (master commit c8f4af6) 1. **Column-substrate identity** (consolidation doc lines 126-220) — the load-bearing architectural reason MultiLaneColumn exists. The full Lance / Arrow / ndarray-SoA diagram + the three load-bearing passages ("one physical representation, end to end", "ndarray amortises the SIMD primitive across the whole stack", "the column IS the SoA IS the ndarray buffer") are reproduced inline in § "Why this exists". 2. **Sprint sequencing** (execution prompt lines 76-91) — the 8-week schedule GridLake patches. Q-NEW-1 marker calls out the two viable insertion points (path a: W2.5 prerequisite slot; path b: X10-A13/A14 absorption) with trade-offs and a recommendation, leaves the decision to the plan-review savant at preflight. 3. **W1 PR-X10 kickoff format** (execution prompt lines 91-144) — mirrored exactly in the final § "Sprint kickoff — W2.5 (path a) or W1-W2 inline (path b)" block. READ FIRST + WORKER DECOMPOSITION + PROTOCOL A + ACCEPTANCE GATES + PR FORMAT + BUDGET + NEXT SPRINTS all present. 4. **Canary gates** (canary plan lines 64-127) — inherited as GridLake's done criteria, with each canary gate mapped to a specific GridLake primitive contribution (e.g., A2's pad_to_lanes invariant gates Correctness.1 bit-exact; A3's bench_no_alloc gates Performance.6 working set ≤ 1 MB; sentinel-qa unsafe audit gates Inhabitance.3 zero P0 SAFETY). ## Negative constraints § "Forbidden" carries the verbatim heel_f64x8 absorption note from pr-x10-linalg-core-design.md, with the directive: if a GridLake SIMD-staged inner loop wants a distance kernel, it calls crate::hpc::linalg::distance::l2_f64_simd (lands in W2 as PR-X10 A6), not a duplicate in crate::simd_soa::*. Plus four more negative constraints: no new SIMD types, no Lance file I/O, no cross-column compute, no runtime schema registry. ## Worker decomposition (4 max-fan-out) - A1 (sequential) — MultiLaneColumn carrier with 7 lane iterators (u8x64 / u16x32 / u32x16 / u64x8 / f32x16 / f64x8 / bf16x32) - A2 (parallel) — simd-soa-derive proc-macro crate (#[derive(SoA)] + #[soa(pad_to_lanes=N)] with strict invariant that pad_to_lanes does NOT change len() semantics) - A3 (parallel) — bench harness (bench_no_alloc + throughput vs raw chunks_exact + lane-width swap cache friendliness + Arc clone O(1)) - A4 (parallel) — integration probes (splat3d backward-compat + nars 10k-seed scalar-vs-SoA parity + Arrow round-trip zero-copy) ## Why this is its own pre-sprint and not part of PR-X10 The Q-NEW-1 framing is honest: path (b) absorbs GridLake into PR-X10 as A13/A14 and saves 0.5 weeks, but the doc recommends path (a) because column-substrate identity is a different concern from linalg primitives under the same coordinator. The plan-review savant decides at preflight; both paths are documented end-to-end so either choice can spawn immediately. ## Status - File lives at .claude/knowledge/hhtl-gridlake-pre-sprint-prompt.md - Fresh branch claude/gridlake-pre-sprint-prompt off post-#162 master - No PR opened — awaiting user instruction
…+ bridge Companion to the GridLake prompt (ade8edb). Names the four current duplicate column-access patterns (BindSpace Box<[u64]>, lance-graph query HashMap<String, RecordBatch>, planner Morsel placeholder enum, lance::Dataset.scan rolled per feature) and the contract crate already exists (lance-graph-contract, 10+ consumers) but lacks a column module. PR-X14′ adds a column/ module set to the existing lance-graph-contract crate (preserving its zero-dep invariant — adds ndarray path dep only; no arrow/lance/datafusion in the contract crate) and a new sibling lance-graph-contract-bridge crate that bridges Lance + Arrow into MultiLaneColumn. SQL path + lance-graph's datafusion deps stay untouched per the 2026-05-19 deferred-SQL decision. Q-NEW-2 marker: plan-review savant picks path α (concurrent with GridLake at W2.5, 10 workers) vs path β (sequential at W3, 4 workers + 0.5 week schedule extension). File evidence cited verbatim from current tree: - lance-graph-contract/src/lib.rs:1-12 (zero-dep claim) - cognitive-shader-driver/src/bindspace.rs:36-44 (Pattern 1) - lance-graph/src/query.rs:243 + sql_query.rs:69 (Pattern 2) - lance-graph-planner/src/physical/mod.rs:37-58 (Pattern 3 placeholder) - holograph + lance-graph-ontology optional lance deps (Pattern 4)
Adds a sibling shopping-list entry below the 2026-05-18 PR-X10 / jc entry, pointing to the two pre-sprint prompts committed earlier on this branch (GridLake ade8edb + PR-X14′ 56b2671) and naming the substrate-pair as the storage-and-contract companion to PR-X10's arithmetic substrate. Records the three architectural decisions that pinned the scope this session: 1. Skip SQL for now (sea-orm contract, sqlparser-rs, PostgREST, fusion parser all deferred) 2. Don't evict datafusion (lance-graph upstream dep, leave alone) 3. Use storage format as-is (no Arrow extension types, no Lance encoder changes) Records two superseded earlier-session proposals (datafusion eviction; two new top-level crates) and the two open Q-markers (Q-NEW-1 GridLake slot, Q-NEW-2 X14′ concurrent vs sequential) for the plan-review savant. Adopts the clutter-replacement-count framework for cost-of-ownership. Cross-references updated with the two new prompt files.
This was referenced May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three commits on this branch capture the storage-and-contract substrate pair that companions the PR-X10 arithmetic substrate, designed in a 2026-05-19 session post PR #162 merge.
ade8edb2— GridLake pre-sprint prompt (PR-X1 carrierMultiLaneColumn+ PR-X2#[derive(SoA)])56b26716— PR-X14′ pre-sprint prompt (lance-graph-contract::columnmodule set + newlance-graph-contract-bridgesibling crate)ee793a74—pr-arithmetic-inventory.md§ 2026-05-19 shopping-list addendum tying both to the existing PR-X10 / jc roadmapWhat the substrate-pair does
The arithmetic substrate (PR-X10) consolidates
crate::hpc::linalg::*and unblocks splat3d/inference/jc-Pillars. The storage-and-contract substrate (PR-X1 + PR-X2 + PR-X14′) is its companion: a singleMultiLaneColumncarrier with typed lane views, an extendedlance-graph-contract::columntrait surface for all consumers, and a sibling bridge crate that wrapslance::Datasetandarrow::RecordBatchinto the carrier.Cost-of-ownership (clutter-replacement count)
The four duplicate patterns cited in the X14′ prompt with verbatim file:line refs:
Box<[u64]>× 4 fingerprint planes —cognitive-shader-driver/src/bindspace.rs:36-44HashMap<String, RecordBatch>—lance-graph/src/query.rs:243,sql_query.rs:69Morsel { columns: Vec<ColumnData /* placeholder */> }—lance-graph-planner/src/physical/mod.rs:37-58 (explicit "placeholder for Arrow integration" comment)lance::Dataset.scan()rolled per optional feature —holograph+lance-graph-ontologyThree architectural decisions that pinned the scope
lance::Datasetandarrow::RecordBatchas-is; no Arrow extension types, no Lance encoder changes, no in-place schema mutation.Forbidden constraints recorded in the prompts
arrow,arrow-schema,arrow-array,lance,lancedb, ordatafusiontolance-graph-contract's Cargo.toml — preserves thelib.rs:1 zero-dependency invariant for the 10+ existing contract consumerslance-graph/src/sql_query.rs,lance-graph/src/sql_catalog.rs, orlance-graph/Cargo.toml's datafusion depsOpen Q-markers for the plan-review savant at preflight
Test plan
This PR is docs-only (three knowledge files added/updated; no source changes).
pr-arithmetic-inventory.mdis preserved unchanged below the new 2026-05-19 addendum (append-only)hhtl-substrate-execution-prompt.mdW1-W8 schedule with the chosen slotCross-references
.claude/knowledge/hhtl-gridlake-pre-sprint-prompt.md(added inade8edb2).claude/knowledge/hhtl-pr-x14-substrate-contract-prompt.md(added in56b26716).claude/knowledge/pr-arithmetic-inventory.md(updated inee793a74, append-only addendum).claude/knowledge/stack-consolidation-bardioc-to-hhtl.md§ "Column-substrate identity" (existing, source-of-truth for the design).claude/knowledge/hhtl-substrate-execution-prompt.md(existing W1-W8 master schedule; will need amendment after Q-marker decisions)Generated by Claude Code