Skip to content

feat: schedule ingestion engine#31

Open
chiptus wants to merge 6 commits intomainfrom
feat/schedule-ingestion
Open

feat: schedule ingestion engine#31
chiptus wants to merge 6 commits intomainfrom
feat/schedule-ingestion

Conversation

@chiptus
Copy link
Copy Markdown
Owner

@chiptus chiptus commented May 9, 2026

Replaces the old client-side CSV import with a server-side ingestion system.

Two Supabase Edge Functions (diff-schedule, commit-schedule) handle the diff and atomic commit via a Postgres RPC. The frontend wizard walks admins through upload → conflict resolution → commit.

Key design decisions: sets matched by artist roster + stage (preserving votes), orphaned sets surfaced as explicit archive/keep conflicts, stage name mismatches resolved via map-to-existing or create-new, all writes wrapped in a single transaction with full rollback on failure.

@vercel
Copy link
Copy Markdown

vercel Bot commented May 9, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
upline Ready Ready Preview, Comment May 9, 2026 3:42pm

chiptus and others added 2 commits May 9, 2026 18:27
Implements the dry-run diff engine for the new schedule ingestion system.
Compares a CSV payload against current DB state for a festival edition and
returns clean operations + conflicts requiring user resolution (orphaned sets,
stage name mismatches). Core business logic extracted into diff.ts with 22
unit tests covering slugging, timezone conversion, B2B matching, and midnight crossing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds the atomic write path for the schedule ingestion system:
- Migration: adds UNIQUE constraints on artists.slug and stages(festival_edition_id, name),
  creates the commit_schedule PL/pgSQL RPC that wraps all writes (artist upserts,
  stage upserts, set inserts/updates, set_artists sync, orphan archiving) in a
  single transaction with full rollback on failure.
- Edge Function: thin admin-gated HTTP handler that calls the RPC via service role key.
- Integration tests for the RPC covering create, update, archive, and time storage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the legacy client-side CSV schedule import with a server-driven ingestion workflow backed by Supabase Edge Functions and a transactional Postgres RPC, plus a new admin import wizard UI (upload → diff/conflicts → commit).

Changes:

  • Added diff-schedule and commit-schedule Edge Functions, plus a commit_schedule Postgres RPC to perform atomic schedule writes.
  • Implemented a new admin schedule import wizard UI with stage-mismatch and orphan-set resolution flows.
  • Added unit tests for diffing logic and integration tests for the commit RPC.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
vite.config.ts Adds test runner config exclusions.
supabase/migrations/20260509142022_commit_schedule_rpc.sql Adds constraints and the transactional commit_schedule RPC used by ingestion.
supabase/functions/_shared/auth.ts Shared admin auth + CORS helpers for Edge Functions.
supabase/functions/diff-schedule/index.ts Edge Function endpoint to compute a diff from CSV rows vs DB.
supabase/functions/diff-schedule/diff.ts Core diff/matching logic (artists, stages, sets, orphan detection).
supabase/functions/diff-schedule/diff.test.ts Unit tests covering slugging, time conversion, matching rules, and conflicts.
supabase/functions/commit-schedule/index.ts Edge Function endpoint that calls the commit_schedule RPC.
supabase/functions/commit-schedule/commit-schedule.test.ts Integration tests targeting the RPC behavior against local Supabase.
src/services/scheduleImportService.ts Frontend service layer for parsing CSV + invoking diff/commit + building commit payloads.
src/pages/admin/FestivalScheduleImport.tsx New admin page wrapper for the import wizard route.
src/pages/admin/FestivalEdition.tsx Adds an “Import” tab and routing to the new import page.
src/components/router/GlobalRoutes.tsx Wires the /import sub-route under festival edition admin routes.
src/components/Admin/ScheduleImport/ScheduleImportWizard.tsx Wizard state machine: upload → review → commit result, plus cache invalidation.
src/components/Admin/ScheduleImport/CsvUploadStep.tsx CSV upload + timezone selection + invokes diff.
src/components/Admin/ScheduleImport/DiffReviewStep.tsx Review UI container including conflicts and commit action.
src/components/Admin/ScheduleImport/DiffSummaryBanner.tsx Summary banner for diff results.
src/components/Admin/ScheduleImport/StageMismatchResolver.tsx UI to map mismatched stage names or create new stages.
src/components/Admin/ScheduleImport/OrphanedSetsPanel.tsx UI to archive/keep orphaned sets not present in CSV.
src/components/Admin/ScheduleImport/CommitResultCard.tsx Success UI and “import another file” reset action.

Comment on lines +60 to +85
UPDATE sets
SET
name = v_set_elem->>'name',
description = NULLIF(v_set_elem->>'description', ''),
stage_id = (
SELECT s.id FROM stages s
WHERE s.festival_edition_id = p_festival_edition_id
AND s.name = v_set_elem->>'stageName'
LIMIT 1
),
time_start = CASE
WHEN (v_set_elem->>'timeStart') IS NOT NULL
THEN (v_set_elem->>'timeStart')::TIMESTAMPTZ
ELSE NULL
END,
time_end = CASE
WHEN (v_set_elem->>'timeEnd') IS NOT NULL
THEN (v_set_elem->>'timeEnd')::TIMESTAMPTZ
ELSE NULL
END,
updated_at = NOW()
WHERE id = (v_set_elem->>'id')::UUID
AND festival_edition_id = p_festival_edition_id;

v_sets_updated := v_sets_updated + 1;

Comment on lines +84 to +93
v_sets_updated := v_sets_updated + 1;

-- Sync set_artists: delete existing links and re-insert from CSV
DELETE FROM set_artists WHERE set_id = (v_set_elem->>'id')::UUID;

INSERT INTO set_artists (set_id, artist_id)
SELECT (v_set_elem->>'id')::UUID, a.id
FROM jsonb_array_elements_text(v_set_elem->'artistSlugs') AS slug_val
JOIN artists a ON a.slug = slug_val
ON CONFLICT (set_id, artist_id) DO NOTHING;
Comment on lines +135 to +137
SELECT v_new_set_id, a.id
FROM jsonb_array_elements_text(v_set_elem->'artistSlugs') AS slug_val
JOIN artists a ON a.slug = slug_val;

const { data: sets } = await db
.from("sets")
.select("time_start, time_end, set_artists(artist_id, artists(slug))")
Comment on lines +66 to +68
// Cleanup
await db.from("artists").delete().eq("slug", slug);
});
Comment on lines +72 to +73
queryClient.invalidateQueries({ queryKey: ["sets", festivalEditionId] });
queryClient.invalidateQueries({ queryKey: ["stages", festivalEditionId] });
Comment on lines +22 to +27
export function CsvUploadStep({ festivalEditionId, onDiffReady }: Props) {
const fileRef = useRef<HTMLInputElement>(null);
const [timezone, setTimezone] = useState("Europe/Lisbon");
const [fileName, setFileName] = useState<string | null>(null);
const [rows, setRows] = useState<CsvRow[]>([]);
const [loading, setLoading] = useState(false);
Comment on lines +51 to +54
<RadioGroupItem value="map" id={`map-${mismatch.csvValue}`} className="mt-0.5" />
<div className="flex-1 space-y-1.5">
<Label htmlFor={`map-${mismatch.csvValue}`} className="cursor-pointer">
Map to existing stage
Comment on lines +79 to +82
<RadioGroupItem value="create" id={`create-${mismatch.csvValue}`} />
<Label htmlFor={`create-${mismatch.csvValue}`} className="cursor-pointer">
Create new stage <span className="font-normal text-muted-foreground">"{mismatch.csvValue}"</span>
</Label>
chiptus and others added 4 commits May 9, 2026 18:35
Adds the full schedule ingestion frontend:
- ScheduleImportWizard: 3-step flow (upload → review → result)
- CsvUploadStep: file drop zone, timezone picker, CSV parse + diff call
- DiffSummaryBanner: counts for new artists/stages/sets/conflicts
- StageMismatchResolver: map-to-existing (dropdown) or create-new per mismatch
- OrphanedSetsPanel: per-set archive/keep toggle, bulk action, default keep
- scheduleImportService: CSV parser, Edge Function callers, commit payload builder
- Import tab added to FestivalEdition page, route wired in GlobalRoutes

Refactors diff.ts SetPayload to use stageName instead of stage_id so
the payload aligns directly with what the commit RPC expects.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extracts DiffReviewStep and CommitResultCard from ScheduleImportWizard to
keep all components under 150 lines per codebase conventions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevents Vitest from picking up supabase/functions Deno tests and
tests/e2e Playwright specs, which caused import errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Deletes CSVImportDialog and csvImportService (326 lines of client-side
import logic). Import CSV buttons removed from StageManagement and
SetManagement — replaced by the dedicated Import tab on the edition page.
parseCSV inlined into scheduleImportService to remove the dependency.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@chiptus chiptus force-pushed the feat/schedule-ingestion branch from 4f1b288 to c8110dd Compare May 9, 2026 15:41
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 9, 2026

DB Migrate failed for stagingworkflow run.

Comment on lines +36 to +52
const reader = new FileReader();
reader.onload = (ev) => {
const content = ev.target?.result as string;
try {
const parsed = parseScheduleCsv(content);
if (parsed.length === 0) {
setError("No valid rows found. Make sure your CSV has an 'Artists' column.");
setRows([]);
} else {
setRows(parsed);
}
} catch {
setError("Failed to parse CSV. Check the file format.");
setRows([]);
}
};
reader.readAsText(file);
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an async function (with the onload), create a "readFile" async function, which takes a file parameter and use useMutations here and in other places that use async functions

}

return (
<div className="space-y-6">
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

break this jsx into smaller logical components

export const Route = createFileRoute(
"/admin/festivals/$festivalSlug/editions/$editionSlug/import",
)({
component: FestivalScheduleImport,
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

co locate the route component

from: "/admin/festivals/$festivalSlug/editions/$editionSlug/import",
});

const editionQuery = useFestivalEditionBySlugQuery({ festivalSlug, editionSlug });
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why useQuery and not get from the route context?

@@ -0,0 +1,196 @@
import { supabase } from "@/integrations/supabase/client";

function parseCSV(csvContent: string): string[][] {
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use a library for this?

return new Date(naiveUtc.getTime() + offsetMs).toISOString();
}

export function computeDiff(
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

break this function into smaller logical function, make this function clear to read

-- RPC: commit_schedule
-- Executes a fully resolved schedule import inside a single transaction.
-- Called by the commit-schedule Edge Function using the service role key.
CREATE OR REPLACE FUNCTION public.commit_schedule(
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we create additional helper function so this function is easier to read?

</div>

{mismatches.map((mismatch) => {
const resolution = resolutions[mismatch.csvValue] ?? {
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

break into a MismachItem component (and it should probably be composed of smaller components too)

const [orphanResolutions, setOrphanResolutions] = useState<
Record<string, OrphanResolution>
>({});
const [committing, setCommitting] = useState(false);
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we use useMutation?

const time = formatTime(set.timeStart);

return (
<div key={set.id} className="flex items-center justify-between px-4 py-3">
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

break into OrphanedItem component

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants