Skip to content

SPOC-513: Restore a usable upgrade chain from 5.x to 6.0.0-devel on main#425

Draft
danolivo wants to merge 2 commits intomainfrom
spoc-upgrade-507-devel
Draft

SPOC-513: Restore a usable upgrade chain from 5.x to 6.0.0-devel on main#425
danolivo wants to merge 2 commits intomainfrom
spoc-upgrade-507-devel

Conversation

@danolivo
Copy link
Copy Markdown
Contributor

Motivation

On main there was no base install script and no upgrade path from any 5.x release — the only SQL install was spock--6.0.0-devel.sql. Disabling that script (e.g. to simulate an ALTER EXTENSION from a 5.x node) produced:

ERROR: extension "spock" has no installation script nor update path for version "6.0.0-devel"

Meanwhile v5_STABLE already shipped spock--5.0.0.sql as a base and a 5.0.6 -> 5.0.7 upgrade. main needs to meet that chain so a site running 5.0.7 on v5_STABLE has a supported path forward.

Changes

  • Added sql/spock--5.0.0.sql — base install taken from v5_STABLE.
  • Added sql/spock--5.0.6--5.0.7.sql — taken from v5_STABLE.
  • Added sql/spock--5.0.7--6.0.0-devel.sql — new. Derived from the previous 5.0.6--6.0.0-devel.sql minus the pause_apply_workers / resume_apply_workers CREATEs, which already exist in 5.0.7 with identical signatures and C symbols.
  • Removed sql/spock--5.0.6--6.0.0-devel.sql — superseded by the two-step 5.0.6 -> 5.0.7 -> 6.0.0-devel path.
    Post-change chain:
    5.0.0 -> 5.0.1 -> ... -> 5.0.6 -> 5.0.7 -> 6.0.0-devel

Fix for sub_skip_schema catalogue-type mismatch

The column was added as text in sql/spock--5.0.1--5.0.2.sql, but the C code has always treated it as text[] on both paths:

  • Write: PointerGetDatum(strlist_to_textarray(sub->skip_schema)) (src/spock_node.c, two sites)
  • Read: DatumGetArrayTypeP(d) + textarray_to_list(...) (src/spock_node.c)
    Because heap_form_tuple / heap_getattr don't validate Datum-vs-column type and both text and ArrayType are varlena, the mismatch ran silently: the bytes stored in the text column were, in fact, a valid ArrayType. The feature worked end-to-end; only SQL-level queries against the column saw garbage (\x01...), which the regression suite exposed via alter_options.
    sql/spock--5.0.7--6.0.0-devel.sql now relabels the column in place:

@danolivo danolivo self-assigned this Apr 21, 2026
@danolivo danolivo added the bug Something isn't working label Apr 21, 2026
@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented Apr 21, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics -2 duplication

Metric Results
Duplication -2

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 21, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d6f6865b-8bce-4bc0-8f1a-54b71a92d497

📥 Commits

Reviewing files that changed from the base of the PR and between 3b01731 and 275615e.

📒 Files selected for processing (4)
  • sql/spock--5.0.0.sql
  • sql/spock--5.0.6--5.0.7.sql
  • sql/spock--5.0.7--6.0.0-devel.sql
  • src/spock_apply_heap.c
💤 Files with no reviewable changes (1)
  • src/spock_apply_heap.c
✅ Files skipped from review due to trivial changes (2)
  • sql/spock--5.0.6--5.0.7.sql
  • sql/spock--5.0.7--6.0.0-devel.sql
🚧 Files skipped from review as they are similar to previous changes (1)
  • sql/spock--5.0.0.sql

📝 Walkthrough

Walkthrough

Adds a full Spock extension install script (v5.0.0) defining catalog tables, views, aggregates, and many SQL/C/PLpgSQL API functions; introduces two upgrade scripts (5.0.6→5.0.7 and 5.0.7→6.0.0-devel) that add/remove pause/resume functions and relabel a catalog column; removes internal apply helpers from src/spock_apply_heap.c.

Changes

Spock Extension Installation (v5.0.0)

Layer / File(s) Summary
Header / Guard
sql/spock--5.0.0.sql
File introduces extension install script and declarations.
Catalog Tables / Views
sql/spock--5.0.0.sql
Adds spock.node, spock.node_interface, spock.local_node, spock.subscription, spock.local_sync_status, spock.exception_*, spock.progress, spock.replication_set*, spock.sequence_state, spock.queue, and supporting metadata tables plus views like spock.TABLES, spock.channel_table_stats, spock.channel_summary_stats, spock.lag_tracker.
Functions / Procedures (API)
sql/spock--5.0.0.sql
Declares numerous SQL/PLpgSQL/C functions and procedures for node/subscription/repset management, table/sequence membership, partitioning, DDL replication, inspection/status, sync orchestration (sub_wait_for_sync, table_wait_for_sync, sync_event, wait_for_sync_event), and monitoring helpers.
Aggregates / Utilities
sql/spock--5.0.0.sql
Adds spock.md5_agg aggregate and channel/stat helper functions, plus read-only helpers like spock.terminate_active_transactions.
Version-gated Helpers / PL/pgSQL
sql/spock--5.0.0.sql
Adds PL/pgSQL DO block that defines convert_column_to_int8/convert_sequence_to_snowflake for PG majors 15–18 and raises on other majors.
Delta/Repair/Worker APIs
sql/spock--5.0.0.sql
Declares C-backed delta_apply functions for numeric types, repair/LSN helpers, spock.get_apply_worker_status, and spock.wait_for_apply_worker.

Spock Extension Upgrade (5.0.6 → 5.0.7)

Layer / File(s) Summary
Header / Guard
sql/spock--5.0.6--5.0.7.sql
Adds script header and psql-sourcing guard that instructs to use ALTER EXTENSION to update.
New Functions
sql/spock--5.0.6--5.0.7.sql
Creates two C-backed VOLATILE functions in spock schema: spock.pause_apply_workers() and spock.resume_apply_workers() mapped to spock_pause_apply_workers / spock_resume_apply_workers.
Privilege Adjustment
sql/spock--5.0.6--5.0.7.sql
Revokes EXECUTE privilege on both functions from PUBLIC.

Spock Extension Upgrade (5.0.7 → 6.0.0-devel)

Layer / File(s) Summary
Header / Guard
sql/spock--5.0.7--6.0.0-devel.sql
Updates script identification/comment and keeps the psql-sourcing guard.
Catalog Correction
sql/spock--5.0.7--6.0.0-devel.sql
Performs in-place UPDATE pg_catalog.pg_attribute to relabel spock.subscription.sub_skip_schema to type text[] and sets attndims = 1.
Upgrade Script Cleanup
sql/spock--5.0.7--6.0.0-devel.sql
Removes the CREATE FUNCTION/REVOKE statements that would recreate spock.pause_apply_workers() / spock.resume_apply_workers() and replaces them with a comment noting those functions already exist as of 5.0.7.

Apply heap implementation cleanup

Layer / File(s) Summary
Core Implementation Removal
src/spock_apply_heap.c
Removes internal helper fill_missing_defaults() that computed default expressions for unmapped physical columns.
Executor Lifecycle Helpers Removed
src/spock_apply_heap.c
Deletes init_apply_exec_state() and finish_apply_exec_state() that initialized/teardown ApplyExecState (EPQ, AFTER triggers, index/result relation handling, tuple-table cleanup).
Inline Replacement (call sites updated in same file)
src/spock_apply_heap.c
Insert/update/delete routines now handle EPQ/trigger/index setup and cleanup inline using create_edata_for_relation()/finish_edata() and explicit Exec/EPQ calls (changes localized to implementation).

Poem

🐇 I nibble at SQL and C with cheer,
New tables sprout and functions appear.
Pause and resume, relabel a stream,
I chase LSNs in a midnight dream.
Tiny paws, big changes—hop! 🥕

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The PR title accurately reflects the main objective: restoring an upgrade chain from 5.x to 6.0.0-devel on main, which is the primary change across all modified files.
Description check ✅ Passed The description comprehensively explains the motivation, all changes made (SQL scripts added/removed), the resulting upgrade chain, and a catalog-type fix for sub_skip_schema with clear rationale.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch spoc-upgrade-507-devel

Warning

Review ran into problems

🔥 Problems

Timed out fetching pipeline failures after 30000ms


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
sql/spock--5.0.7--6.0.0-devel.sql (1)

6-19: Catalog relabel is correct; consider adding a defensive predicate for idempotency.

The approach is sound: text and text[] share identical physical storage attributes (typlen=-1, typbyval=false, typalign='i', typstorage='x'), so attlen / attbyval / attalign / attstorage need not change. Existing on-disk values were already written via strlist_to_textarray (src/spock_node.c:903-907) and read via DatumGetArrayTypeP (src/spock_node.c:1129-1137), so simply relabeling atttypid and setting attndims = 1 fixes SQL-level access (SELECT, unnest, array ops) without tuple rewrites, and the UPDATE triggers normal relcache invalidation on commit.

One hardening suggestion: scope the UPDATE to the exact prior state so the statement is a strict no-op if re-run or if a site has already been patched:

Proposed defensive predicate
 UPDATE pg_catalog.pg_attribute
    SET atttypid = 'text[]'::regtype,
        attndims = 1
  WHERE attrelid = 'spock.subscription'::regclass
-   AND attname  = 'sub_skip_schema';
+   AND attname  = 'sub_skip_schema'
+   AND atttypid = 'text'::regtype;

Direct DML on pg_catalog.pg_attribute requires superuser privileges, which ALTER EXTENSION … UPDATE provides, so no GUC adjustment is needed.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@sql/spock--5.0.7--6.0.0-devel.sql` around lines 6 - 19, The UPDATE should be
made idempotent by restricting it to rows that still have the old catalog state;
modify the UPDATE on pg_catalog.pg_attribute (targeting spock.subscription,
attname = 'sub_skip_schema') to only set atttypid = 'text[]'::regtype and
attndims = 1 when the current atttypid equals 'text'::regtype (and/or attndims =
0), so the statement becomes a no-op if the column is already relabeled.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@sql/spock--5.0.7--6.0.0-devel.sql`:
- Around line 6-19: The UPDATE should be made idempotent by restricting it to
rows that still have the old catalog state; modify the UPDATE on
pg_catalog.pg_attribute (targeting spock.subscription, attname =
'sub_skip_schema') to only set atttypid = 'text[]'::regtype and attndims = 1
when the current atttypid equals 'text'::regtype (and/or attndims = 0), so the
statement becomes a no-op if the column is already relabeled.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a0de3266-3351-4286-9c28-96102bad964d

📥 Commits

Reviewing files that changed from the base of the PR and between a4b93e8 and 27362fc.

📒 Files selected for processing (3)
  • sql/spock--5.0.0.sql
  • sql/spock--5.0.6--5.0.7.sql
  • sql/spock--5.0.7--6.0.0-devel.sql

@danolivo danolivo requested review from mason-sharp and rasifr April 22, 2026 07:59
@danolivo danolivo force-pushed the spoc-upgrade-507-devel branch from c6bdcfc to 3b01731 Compare April 22, 2026 07:59
danolivo added 2 commits May 4, 2026 10:01
On main there was no base install file and no upgrade path from 5.x,
so CREATE EXTENSION could only land on 6.0.0-devel via its single full
install script.

Disabling that script exposed the gap — ALTER EXTENSION had nothing to walk.
Restore the full chain:
  sql/spock--5.0.0.sql           -- base install (from v5_STABLE)
  sql/spock--5.0.6--5.0.7.sql    -- from v5_STABLE
  sql/spock--5.0.7--6.0.0-devel.sql

The new 5.0.7 -> 6.0.0-devel step is derived from the old 5.0.6 ->
6.0.0-devel script minus the pause/resume_apply_workers
function definitions, which already exist in 5.0.7 with identical
signatures and C symbols.

It also fixes a pre-existing catalogue-type mismatch on
spock.subscription.sub_skip_schema.  The column was added as text in
the 5.0.1 -> 5.0.2 upgrade, but the C code on both read and write has
always treated it as text[] (strlist_to_textarray on write,
DatumGetArrayTypeP on read).  The feature worked because the bytes on
disk were a valid ArrayType all along; only SQL-level queries against
the column saw garbage (\x01...).  Relabel the column in place via
pg_attribute so SELECT / unnest / array operators work as expected,
without rewriting any tuples.
@danolivo danolivo force-pushed the spoc-upgrade-507-devel branch from 3b01731 to 275615e Compare May 4, 2026 08:07
@danolivo danolivo marked this pull request as draft May 5, 2026 16:09
@danolivo danolivo added the in draft This PR created to test domething label May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working in draft This PR created to test domething

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant