Skip to content

feat(appkit): vendor TaskFlow engine and add TaskflowService skeleton#376

Draft
ditadi wants to merge 1 commit into
stack/taskflow/sql-warehouse-splitfrom
stack/taskflow/taskflow-service
Draft

feat(appkit): vendor TaskFlow engine and add TaskflowService skeleton#376
ditadi wants to merge 1 commit into
stack/taskflow/sql-warehouse-splitfrom
stack/taskflow/taskflow-service

Conversation

@ditadi
Copy link
Copy Markdown
Contributor

@ditadi ditadi commented May 11, 2026

🥞 Stacked PR

Use this link to review incremental changes.


Bootstraps a third core service (this.taskflow, peer of this.cache and this.telemetry) backed by the vendored Rust+napi engine at packages/appkit/vendor/taskflow/. The service is wired into the CoreServiceRegistry introduced in #374 so existing createApp(...) callers transparently gain durable execution plumbing, with an opt-out via createApp({ taskflow: false }).

Scope of this PR (engine plumbing only)

  • Vendored binaries (taskflow-darwin-arm64.node, taskflow-linux-x64.node), the JS loader, type declarations, and VENDOR.json with SHA-256 digests for the optional integrity check (APPKIT_VERIFY_TASKFLOW_VENDOR=1).
  • TaskflowService wraps the vendor Engine with lifecycle hooks (initialize, factory, shutdown) and pass-through methods (start, subscribe, reconnect, resume, stop, simulateCrash). Default storage: SQLite at .appkit/taskflow/. Emits a warn on Databricks Apps + SQLite (multi-pod / ephemeral filesystem; tasks would not survive rolling restarts).
  • Plugin base class exposes this.taskflow (TaskflowService | null) and requireTaskflow() for the strict-mode caller. The deprecated taskflow legacy field on BasePlugin.attachContext lands here, paired with the same deprecation flags for cache? / telemetry? that the registry restructure (refactor(appkit): introduce ServiceManager for core service lifecycle #374) already implied.
  • ServerPlugin._gracefulShutdown now consumes the shutdownCoreServices callback that _createApp already passes, draining cache/telemetry/taskflow through the registry. The SIGTERM/SIGINT re-entrancy guard is intentionally deferred to the hardening pass.
  • tsdown.config.ts copies the vendor files into dist/appkit/vendor/ so the published package's emitted loader resolves the .node binary at the expected relative path.
  • .gitignore (root + dev-playground + template) excludes the per-machine SQLite/WAL files under .appkit/.
  • knip.json skips packages/appkit/vendor/** (vendored module exports are not consumed by AppKit's own code).

Deliberately NOT in this PR (follow-up)

  • task() registration, TaskDefinition, TypedTaskContext, TaskRef, the step() helper.
  • Plugin.executeTask SSE bridge (the bridge file, ActiveBridge, _registerBridge, the shutdown drain loop).
  • Re-exports for SseEvent, TASKFLOW_IK_HEADER, setupSseHeaders, writeSseFrame, ExecuteTaskSettings.
  • The tools/test-helpers.ts stub service + SSE parser upgrade.
  • Analytics-plugin migration; durable-task example app.

Plugins can call this.taskflow.start(name, input) and subscribe via this.taskflow.subscribe(idempotencyKey), but no task can be defined yet — task() registration arrives with the typed surface in the next PR.

Verified: pnpm -r typecheck, pnpm build, pnpm exec biome check, pnpm exec knip, full pnpm test (122 files, 2279 tests; +3 from #375) all green.

Signed-off-by: ditadi victordperd@gmail.com

Bootstraps a third core service (`this.taskflow`, peer of `this.cache`
and `this.telemetry`) backed by the vendored Rust+napi engine at
`packages/appkit/vendor/taskflow/`. The service is wired into the
`CoreServiceRegistry` introduced in #374 so existing `createApp(...)`
callers transparently gain durable execution plumbing, with an
opt-out via `createApp({ taskflow: false })`.

Scope of this PR (engine plumbing only):

- Vendored binaries (`taskflow-darwin-arm64.node`,
  `taskflow-linux-x64.node`), the JS loader, type declarations, and
  `VENDOR.json` with SHA-256 digests for the optional integrity
  check (`APPKIT_VERIFY_TASKFLOW_VENDOR=1`).
- `TaskflowService` wraps the vendor `Engine` with lifecycle hooks
  (`initialize`, `factory`, `shutdown`) and pass-through methods
  (`start`, `subscribe`, `reconnect`, `resume`, `stop`,
  `simulateCrash`). Default storage: SQLite at `.appkit/taskflow/`.
  Emits a `warn` on Databricks Apps + SQLite (multi-pod / ephemeral
  filesystem; tasks would not survive rolling restarts).
- `Plugin` base class exposes `this.taskflow` (`TaskflowService | null`)
  and `requireTaskflow()` for the strict-mode caller. The deprecated
  `taskflow` legacy field on `BasePlugin.attachContext` lands here,
  paired with the same deprecation flags for `cache?` / `telemetry?`
  that the registry restructure (#374) already implied.
- `ServerPlugin._gracefulShutdown` now consumes the
  `shutdownCoreServices` callback that `_createApp` already passes,
  draining cache/telemetry/taskflow through the registry. The
  SIGTERM/SIGINT re-entrancy guard is intentionally deferred to the
  hardening pass.
- `tsdown.config.ts` copies the vendor files into
  `dist/appkit/vendor/` so the published package's emitted loader
  resolves the `.node` binary at the expected relative path.
- `.gitignore` (root + dev-playground + template) excludes the
  per-machine SQLite/WAL files under `.appkit/`.
- `knip.json` skips `packages/appkit/vendor/**` (vendored module
  exports are not consumed by AppKit's own code).

Deliberately NOT in this PR (follow-up):

- `task()` registration, `TaskDefinition`, `TypedTaskContext`,
  `TaskRef`, the `step()` helper.
- `Plugin.executeTask` SSE bridge (the bridge file, `ActiveBridge`,
  `_registerBridge`, the shutdown drain loop).
- Re-exports for `SseEvent`, `TASKFLOW_IK_HEADER`,
  `setupSseHeaders`, `writeSseFrame`, `ExecuteTaskSettings`.
- The `tools/test-helpers.ts` stub service + SSE parser upgrade.
- Analytics-plugin migration; durable-task example app.

Plugins can call `this.taskflow.start(name, input)` and subscribe via
`this.taskflow.subscribe(idempotencyKey)`, but no task can be defined
yet — `task()` registration arrives with the typed surface in the
next PR.

Verified: pnpm -r typecheck, pnpm build, pnpm exec biome check,
pnpm exec knip, full pnpm test (122 files, 2279 tests; +3 from #375)
all green.

Signed-off-by: ditadi <victordperd@gmail.com>
@ditadi ditadi force-pushed the stack/taskflow/sql-warehouse-split branch from 9e0ea14 to b1491b8 Compare May 12, 2026 17:25
@ditadi ditadi force-pushed the stack/taskflow/taskflow-service branch from 9598bd2 to eebf00c Compare May 12, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant