Skip to content

feat: add first-class CLI interaction modes#143

Merged
nicknisi merged 10 commits into
mainfrom
nick/host-capability-probe
May 12, 2026
Merged

feat: add first-class CLI interaction modes#143
nicknisi merged 10 commits into
mainfrom
nick/host-capability-probe

Conversation

@nicknisi
Copy link
Copy Markdown
Member

@nicknisi nicknisi commented May 4, 2026

Summary

  • Adds first-class interaction modes with global --mode human|agent|ci and WORKOS_MODE, while keeping --json scoped to output formatting.
  • Preserves compatibility for WORKOS_NO_PROMPT, non-TTY JSON defaults, and WORKOS_FORCE_TTY as an output-only override.
  • Surfaces interactionMode in workos doctor and runs host-execution probing for agent/CI mode so sandboxed runs are visible.
  • Migrates auth, login, install routing, API/env/config destructive flows, claim/debug behavior, and structured errors to branch on prompt/agent/CI semantics instead of JSON mode.
  • Adds recovery hints, agent/CI-specific guidance, machine-readable help metadata, and documentation for the two-axis model.

Why

Coding agents and CI need deterministic non-prompt behavior, but --json is only an output contract. Before this, output formatting, TTY detection, prompts, browser launches, and sandbox trust were coupled. That made pseudo-TTY agents and scripts hard to reason about and made host-sandbox failures look like real WorkOS configuration or auth failures.

This PR separates the two axes: output mode controls formatting; interaction mode controls who is driving the CLI and which behaviors are allowed.

Risk

  • Medium blast radius: this touches startup mode resolution plus auth/login/install/API/env/destructive command paths.
  • Compatibility-sensitive: non-TTY still defaults to agent interaction and JSON output, but WORKOS_FORCE_TTY now only forces human output, not human interaction. Use WORKOS_MODE=human for that.
  • Prompt behavior changes are intentional in agent/CI mode: missing args and destructive operations fail with structured guidance instead of prompting.
  • Main mitigation is broad unit coverage plus runtime smoke tests across inferred, explicit agent, explicit CI, explicit human + JSON, and help JSON modes.

Validation

  • pnpm test (138 files, 1776 tests)
  • pnpm typecheck
  • pnpm lint
  • pnpm format:check
  • pnpm build
  • pnpm exec tsx src/bin.ts doctor --json --skip-ai --skip-api --install-dir "$PWD" -> inferred agent / non_tty report; exits 1 here because this repo intentionally lacks WorkOS env vars
  • WORKOS_MODE=agent pnpm exec tsx src/bin.ts doctor --json --skip-ai --skip-api --install-dir "$PWD" -> agent / env
  • pnpm exec tsx src/bin.ts --mode agent doctor --json --skip-ai --skip-api --install-dir "$PWD" -> agent / flag
  • WORKOS_MODE=ci pnpm exec tsx src/bin.ts doctor --json --skip-ai --skip-api --install-dir "$PWD" -> ci / env
  • WORKOS_MODE=human pnpm exec tsx src/bin.ts doctor --json --skip-ai --skip-api --install-dir "$PWD" -> human / env, proving JSON output and interaction mode are separate
  • pnpm exec tsx src/bin.ts --mode agent doctor --help --json -> doctor command subtree

Summary by CodeRabbit

  • New Features

    • Added global --mode / WORKOS_MODE (human|agent|ci); doctor now reports interaction mode and host-execution status.
  • User-Visible Improvements

    • CLI errors include structured recovery hints and clearer, mode-specific messages when prompts aren’t allowed.
    • Browser-launch, filesystem, keychain, and localhost-bind failures surface explicit visible warnings and guidance.
  • Documentation

    • New guidance on output vs interaction modes and agent/CI usage.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 4, 2026

📝 Walkthrough

Walkthrough

Adds interaction-mode resolution and CLI --mode wiring, structured error/recovery hints and output compatibility, a host-capability probe with observeHostFailure, integrations into credential/config stores and proxies, updated command auth/confirmation flows, doctor/reporting changes, and extensive tests and docs.

Changes

Interaction Mode, Output & Recovery

Layer / File(s) Summary
Interaction-mode resolver & state
src/utils/interaction-mode.ts, src/utils/interaction-mode.spec.ts
Adds InteractionMode types, resolver (resolveInteractionMode), module-level state (set/get/resetInteractionModeForTests), predicates (isHumanMode, isAgentMode, isCiMode, isPromptAllowed), and validation/error types.
CLI wiring & help-json
src/bin.ts, src/utils/help-json.ts, src/utils/help-json.spec.ts
Wire resolveInteractionMode at CLI startup, add global --mode option, add extractHelpJsonCommand(argv) to compute help JSON payloads while skipping --mode, and handle InvalidInteractionModeError at startup.
Structured output & recovery
src/utils/output.ts, src/utils/recovery-hints.ts, src/utils/exit-codes.ts, src/utils/command-invocation.ts
Introduce StructuredError and recovery hints model (RecoveryHint/RecoveryHints), human/JSON rendering of recovery hints, authLoginRecovery/confirmationRecovery/missingArgsRecovery, command-format helpers (shellQuoteArg/formatWorkOSCommandArgs), and update exit helpers to infer recovery hints from interaction mode.

Host Probe and Observability

Layer / File(s) Summary
Host probe types & helpers
src/lib/host-probe.ts, src/lib/host-probe.spec.ts
Add HostCapability (includes localhost-bind), HostOperation/HostCapabilityDetails, extend ProbeFailure with metadata, permission-class heuristics, per-session caching and warn gating, formatting helpers, and tests for probe/warn/observe semantics.
Integrations: stores & proxies
src/lib/config-store.ts, src/lib/credential-store.ts, src/lib/credential-proxy.ts
Call `observeHostFailure('home-fs'

Commands, run-with-core & ensure-auth

Layer / File(s) Summary
Commands & confirmation gating
src/commands/*, src/commands/*.spec.ts
Replace non-interactive env checks with isPromptAllowed/isCiMode/isAgentMode, enforce confirmation/tty requirements via exitWithError and recovery hints, and update tests to use interaction-mode test helpers.
Auth flows & runtime
src/lib/ensure-auth.ts, src/lib/run-with-core.ts, src/commands/login.ts, src/commands/claim.ts
Call warnIfSandboxed() from ensure-auth, add CI-agent branching in login/claim, open browsers with { wait:false }, and route browser-launch failures to observeHostFailure.

Doctor & Reporting

Layer / File(s) Summary
Doctor checks, types & output
src/doctor/types.ts, src/doctor/checks/host-execution.ts, src/doctor/index.ts, src/doctor/issues.ts, src/doctor/output.ts
Add HostExecutionInfo/HostExecutionFailure, include interactionMode and hostExecution in DoctorReport, implement checkHostExecution() and emit HOST_EXECUTION_UNTRUSTED when host execution is untrusted; add human output “Interaction Mode” section and tests.
Secret-scan exclusions
src/doctor/checks/auth-patterns.ts, src/doctor/checks/auth-patterns.spec.ts
Add isIgnoredSecretSourcePath to skip test/fixture/eval files from API-key scanning and tests verifying suppression in those paths.

Possible related PRs:

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch nick/host-capability-probe

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 4, 2026

Greptile Summary

This PR separates output formatting from interaction control by introducing a first-class InteractionMode system (human / agent / ci) resolved from --mode, WORKOS_MODE, CI env markers, agent env markers, and TTY state. It migrates all prompt-gating, browser-launch, destructive-operation, and auth-failure paths off isNonInteractiveEnvironment() onto the new interaction mode, adds structured recovery hints to errors, and surfaces sandbox/host-execution status in workos doctor.

  • interaction-mode.ts: new resolution pipeline with prioritized sources; resolveEffectiveOutputMode coerces output to JSON for explicit agent/CI modes while preserving WORKOS_FORCE_TTY semantics.
  • host-probe.ts: new sandbox-detection module; previously-flagged isPermissionError gating on probeHomeFs/probeKeychain is correctly applied.
  • recovery-hints.ts: structured, mode-aware next-step hints attached to auth_required, confirmation_required, and missing_args errors.

Confidence Score: 4/5

Safe to merge after addressing the sandbox warning suppression in JSON mode.

The sandbox/host-execution warning system is the headline feature for agent visibility, but logVisibleWarn guards its console.error with !isJsonMode() and writeLog also skips output in JSON mode. In the typical agent invocation that auto-enables JSON output, the warning is completely silent. All other changes are well-constructed and previously-flagged issues are addressed.

src/utils/debug.ts — the logVisibleWarn implementation is missing a JSON-mode branch to emit a structured warning to stderr.

Important Files Changed

Filename Overview
src/utils/debug.ts Adds logVisibleWarn for sandbox warnings, but never emits to stderr in JSON mode — silencing all host-probe warnings in the exact agent/CI JSON-output context they were designed for.
src/utils/interaction-mode.ts New file introducing InteractionMode resolution — correctly parses --mode from raw argv, handles CI/agent env markers, TTY detection, and priority ordering.
src/lib/host-probe.ts New sandbox-detection module. Previously-flagged issues are fixed. isLikelyHostFailure returns true unconditionally for browser-launch, which can produce false positives on systems without a browser, but is hedged with 'may be' language.
src/utils/output.ts Adds resolveEffectiveOutputMode, StructuredError, and recovery-hint rendering. Previously-missing export is now present.
src/lib/run-with-core.ts Migrates adapter selection from isNonInteractiveEnvironment() to headlessMode = nonHumanMode && isJsonMode(). Browser-launch errors now surface through observeHostFailure.
src/utils/recovery-hints.ts New module providing mode-aware recovery hints. missingArgsRecovery parameter is now correctly typed as `string
src/utils/help-json.ts Adds extractHelpJsonCommand which correctly skips --mode <value> but does not skip other global flags with values.
src/bin.ts Mode resolution is correctly sequenced before yargs with InvalidInteractionModeError caught to emit a structured error.
src/lib/ensure-auth.ts All four auth-failure branches use mode-specific messages; warnIfSandboxed() is called before any auth checks.
src/doctor/checks/host-execution.ts New check that short-circuits in human mode and runs runHostProbe() in agent/CI mode with caching.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["bin.ts startup"] --> B["resolveOutputMode"]
    B --> C["resolveInteractionMode"]
    C --> D{"source?"}
    D -- flag --> E["mode = --mode value"]
    D -- env --> F["mode = WORKOS_MODE"]
    D -- workos_no_prompt --> G["mode = agent"]
    D -- ci_env --> H["mode = ci"]
    D -- agent_env --> I["mode = agent"]
    D -- non_tty --> J["mode = agent"]
    D -- default --> K["mode = human"]
    E & F & G & H & I & J & K --> L["setInteractionMode"]
    L --> M["resolveEffectiveOutputMode"]
    M --> N{"human or non_tty?"}
    N -- yes --> O["keep base output mode"]
    N -- no --> P["force JSON"]
    O & P --> Q["setOutputMode"]
    Q --> R["Command execution"]
    R --> S{"isPromptAllowed?"}
    S -- yes --> T["Interactive prompts / browser"]
    S -- no --> U["exitWithError + recovery hints"]
    R --> V["ensureAuthenticated"]
    V --> W["warnIfSandboxed"]
    W --> X{"probe.ok?"}
    X -- no --> Y["logVisibleWarn (silent in JSON mode)"]
    X -- yes --> Z["Auth check continues"]
Loading

Reviews (11): Last reviewed commit: "feat: coerce output to JSON for explicit..." | Re-trigger Greptile

greptile-apps[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Comment thread src/lib/host-probe.ts
devin-ai-integration[bot]

This comment was marked as resolved.

greptile-apps[bot]

This comment was marked as resolved.

nicknisi and others added 5 commits May 8, 2026 16:52
When the CLI runs inside an AI agent sandbox (Claude Code, Codex, Cursor),
the keyring and home directory may be unavailable. Instead of letting
opaque EPERM errors confuse the agent, we now:

1. Proactively probe home-fs and keychain on first auth check in
   non-interactive mode (warnIfSandboxed in ensure-auth)
2. Reactively observe permission errors in keyring read/write calls
   in both credential-store and config-store (observeHostFailure)
3. Emit a single actionable warning per session pointing the user
   to re-run on the host shell
- probeKeychain() no longer reports the keychain as failed when the
  probe entry is simply absent. A 'not found' / 'No such' error from
  @napi-rs/keyring now indicates a healthy keychain (matches the
  existing pattern in deleteFromKeyring in config-store and
  credential-store), so non-interactive runs on healthy hosts no longer
  emit false-positive sandbox warnings.
- Replace require('@napi-rs/keyring') with a static ES import. The
  package has 'type: module', so the previous require() threw
  ReferenceError at runtime and caused probeKeychain to always fail.
- Switch probeHomeFs to node:fs/promises and make runHostProbe and
  warnIfSandboxed async, per the project's no-sync-fs guideline.
- Tighten the /sandbox/i permission pattern to /\bsandboxd?\b/i so
  unrelated error messages containing 'sandbox' as a substring don't
  trigger sandbox warnings.

Updates the spec to drive the async API and adds coverage for the
healthy-keychain (entry-absent) and substring-collision cases.

Co-Authored-By: nick.nisi@workos.com <nick.nisi@workos.com>
Co-Authored-By: nick.nisi@workos.com <nick.nisi@workos.com>
probeHomeFs previously treated every fs error as a sandbox indicator,
so transient errors like ENOSPC or EIO would emit a misleading
"sandboxed environment" warning. Gate the catch block on
isPermissionError so it stays consistent with observeHostFailure.

Co-Authored-By: nick.nisi@workos.com <nick.nisi@workos.com>
probeKeychain previously treated any non-missing-entry keychain error
as a sandbox indicator. A user-canceled macOS keychain prompt or a
transient keyring daemon error would therefore produce a misleading
"sandboxed environment" warning on healthy hosts. Mirror probeHomeFs
and observeHostFailure by ignoring non-permission errors.

Co-Authored-By: nick.nisi@workos.com <nick.nisi@workos.com>
@nicknisi nicknisi force-pushed the nick/host-capability-probe branch from 1046ad3 to a9b122a Compare May 8, 2026 23:57
devin-ai-integration Bot and others added 2 commits May 9, 2026 00:12
Move fs.unlink into a finally block with best-effort error handling so
a successful writeFile never leaves an orphan probe file in ~/.workos
when unlink itself fails. The probe is checking write access; cleanup
should not affect the result.

Co-Authored-By: nick.nisi@workos.com <nick.nisi@workos.com>
@nicknisi nicknisi changed the title feat: add host-capability probes for sandboxed environments feat: add host execution diagnostics for sandboxed agents May 9, 2026
@nicknisi nicknisi changed the title feat: add host execution diagnostics for sandboxed agents feat: add first-class CLI interaction modes May 9, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ff7cc87c-3f3a-46a9-92dd-6473aaee1be5

📥 Commits

Reviewing files that changed from the base of the PR and between a9b122a and 889c931.

📒 Files selected for processing (51)
  • DEVELOPMENT.md
  • README.md
  • src/bin.ts
  • src/commands/api/index.spec.ts
  • src/commands/api/index.ts
  • src/commands/claim.spec.ts
  • src/commands/claim.ts
  • src/commands/connection.spec.ts
  • src/commands/connection.ts
  • src/commands/debug.spec.ts
  • src/commands/debug.ts
  • src/commands/directory.spec.ts
  • src/commands/directory.ts
  • src/commands/env.spec.ts
  • src/commands/env.ts
  • src/commands/login.spec.ts
  • src/commands/login.ts
  • src/doctor/checks/auth-patterns.spec.ts
  • src/doctor/checks/auth-patterns.ts
  • src/doctor/checks/host-execution.spec.ts
  • src/doctor/checks/host-execution.ts
  • src/doctor/index.ts
  • src/doctor/issues.spec.ts
  • src/doctor/issues.ts
  • src/doctor/json-output.spec.ts
  • src/doctor/output.spec.ts
  • src/doctor/output.ts
  • src/doctor/types.ts
  • src/lib/config-store.ts
  • src/lib/credential-proxy.ts
  • src/lib/credential-store.ts
  • src/lib/ensure-auth.spec.ts
  • src/lib/ensure-auth.ts
  • src/lib/host-probe.spec.ts
  • src/lib/host-probe.ts
  • src/lib/run-with-core.ts
  • src/utils/debug.spec.ts
  • src/utils/debug.ts
  • src/utils/environment.spec.ts
  • src/utils/environment.ts
  • src/utils/exit-codes.spec.ts
  • src/utils/exit-codes.ts
  • src/utils/help-json.spec.ts
  • src/utils/help-json.ts
  • src/utils/interaction-mode.spec.ts
  • src/utils/interaction-mode.ts
  • src/utils/mode-compatibility.spec.ts
  • src/utils/output.spec.ts
  • src/utils/output.ts
  • src/utils/recovery-hints.spec.ts
  • src/utils/recovery-hints.ts
✅ Files skipped from review due to trivial changes (1)
  • src/doctor/json-output.spec.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/host-probe.spec.ts

Comment thread README.md Outdated
Comment thread src/commands/env.ts Outdated
Comment thread src/lib/run-with-core.ts
Comment on lines +26 to +31
it('does not inspect WORKOS_FORCE_TTY directly', () => {
process.env.WORKOS_FORCE_TTY = '1';
setInteractionMode({ mode: 'agent', source: 'non_tty' });
expect(isNonInteractiveEnvironment()).toBe(true);
delete process.env.WORKOS_FORCE_TTY;
});
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Restore previous WORKOS_FORCE_TTY value instead of always deleting it.

Line 30 can leak state when this env var is already set in the test process.

Suggested fix
 it('does not inspect WORKOS_FORCE_TTY directly', () => {
+  const prevForceTty = process.env.WORKOS_FORCE_TTY;
   process.env.WORKOS_FORCE_TTY = '1';
   setInteractionMode({ mode: 'agent', source: 'non_tty' });
   expect(isNonInteractiveEnvironment()).toBe(true);
-  delete process.env.WORKOS_FORCE_TTY;
+  if (prevForceTty === undefined) {
+    delete process.env.WORKOS_FORCE_TTY;
+  } else {
+    process.env.WORKOS_FORCE_TTY = prevForceTty;
+  }
 });
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
it('does not inspect WORKOS_FORCE_TTY directly', () => {
process.env.WORKOS_FORCE_TTY = '1';
setInteractionMode({ mode: 'agent', source: 'non_tty' });
expect(isNonInteractiveEnvironment()).toBe(true);
delete process.env.WORKOS_FORCE_TTY;
});
it('does not inspect WORKOS_FORCE_TTY directly', () => {
const prevForceTty = process.env.WORKOS_FORCE_TTY;
process.env.WORKOS_FORCE_TTY = '1';
setInteractionMode({ mode: 'agent', source: 'non_tty' });
expect(isNonInteractiveEnvironment()).toBe(true);
if (prevForceTty === undefined) {
delete process.env.WORKOS_FORCE_TTY;
} else {
process.env.WORKOS_FORCE_TTY = prevForceTty;
}
});

Comment thread src/utils/help-json.ts
Comment thread src/utils/mode-compatibility.spec.ts
- Use `{ wait: false }` for opn() in device auth to avoid blocking poll
- Disambiguate error messages for CI vs generic non-prompt scenarios
- Skip --mode values in help-json command extraction
- Fix contradictory WORKOS_NO_PROMPT docs in README
Comment thread src/lib/run-with-core.ts
import type { InstallerOptions } from '../utils/types.js';
import { isNonInteractiveEnvironment } from '../utils/environment.js';
import { getInteractionMode, isAgentMode, isCiMode } from '../utils/interaction-mode.js';
import { getOutputMode, isJsonMode, resolveEffectiveOutputMode, setOutputMode } from '../utils/output.js';
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Missing export: resolveEffectiveOutputMode does not exist in output.ts

resolveEffectiveOutputMode is imported from ../utils/output.js here but is never defined or exported from that module. With strict: true and "module": "nodenext" in tsconfig.json, TypeScript will reject this as a compile-time error, and Node.js ESM will throw a hard link-time SyntaxError at startup when this module is first loaded — breaking the entire installer flow for any runWithCore invocation. The function needs to be defined and exported from output.ts.

Comment thread src/commands/env.ts
: isCiMode()
? 'Name and API key required in CI mode.'
: 'Name and API key required when prompting is unavailable.',
recovery: missingArgsRecovery(undefined, 'Provide environment name and API key as positional arguments.'),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Type mismatch: undefined passed to command: string parameter

missingArgsRecovery is declared as (command: string, description: string), but undefined is passed as the first argument. With strictNullChecks: true this is a TypeScript compile error. Since the intent is a description-only hint, the signature of missingArgsRecovery should accept string | undefined, or the call site should pass a concrete example command.

Suggested change
recovery: missingArgsRecovery(undefined, 'Provide environment name and API key as positional arguments.'),
recovery: missingArgsRecovery(
formatWorkOSCommand('env add <name> <api-key> [--client-id <id>]'),
'Provide environment name and API key as positional arguments.',
),

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

♻️ Duplicate comments (1)
src/utils/environment.spec.ts (1)

26-31: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Restore prior WORKOS_FORCE_TTY value instead of always deleting it.

Line 30 can clobber an existing test-process env value and leak state across tests.

Suggested fix
 it('does not inspect WORKOS_FORCE_TTY directly', () => {
+  const prevForceTty = process.env.WORKOS_FORCE_TTY;
   process.env.WORKOS_FORCE_TTY = '1';
-  setInteractionMode({ mode: 'agent', source: 'non_tty' });
-  expect(isNonInteractiveEnvironment()).toBe(true);
-  delete process.env.WORKOS_FORCE_TTY;
+  try {
+    setInteractionMode({ mode: 'agent', source: 'non_tty' });
+    expect(isNonInteractiveEnvironment()).toBe(true);
+  } finally {
+    if (prevForceTty === undefined) delete process.env.WORKOS_FORCE_TTY;
+    else process.env.WORKOS_FORCE_TTY = prevForceTty;
+  }
 });
🧹 Nitpick comments (1)
src/commands/api/index.spec.ts (1)

367-375: ⚡ Quick win

Make the agent-mode stderr assertion less brittle.

Line 374 hard-codes a typo-prone literal (agent mode require --yes). Prefer intent-based matching so copy tweaks don’t cause test churn.

♻️ Suggested change
-    expect(stderrOutput.some((l) => l.includes('agent mode require --yes'))).toBe(true);
+    expect(stderrOutput.some((l) => /agent mode.*--yes/i.test(l))).toBe(true);

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b3a843be-a7d5-428c-bbe3-ae407f3305c3

📥 Commits

Reviewing files that changed from the base of the PR and between a9b122a and e2210d8.

📒 Files selected for processing (51)
  • DEVELOPMENT.md
  • README.md
  • src/bin.ts
  • src/commands/api/index.spec.ts
  • src/commands/api/index.ts
  • src/commands/claim.spec.ts
  • src/commands/claim.ts
  • src/commands/connection.spec.ts
  • src/commands/connection.ts
  • src/commands/debug.spec.ts
  • src/commands/debug.ts
  • src/commands/directory.spec.ts
  • src/commands/directory.ts
  • src/commands/env.spec.ts
  • src/commands/env.ts
  • src/commands/login.spec.ts
  • src/commands/login.ts
  • src/doctor/checks/auth-patterns.spec.ts
  • src/doctor/checks/auth-patterns.ts
  • src/doctor/checks/host-execution.spec.ts
  • src/doctor/checks/host-execution.ts
  • src/doctor/index.ts
  • src/doctor/issues.spec.ts
  • src/doctor/issues.ts
  • src/doctor/json-output.spec.ts
  • src/doctor/output.spec.ts
  • src/doctor/output.ts
  • src/doctor/types.ts
  • src/lib/config-store.ts
  • src/lib/credential-proxy.ts
  • src/lib/credential-store.ts
  • src/lib/ensure-auth.spec.ts
  • src/lib/ensure-auth.ts
  • src/lib/host-probe.spec.ts
  • src/lib/host-probe.ts
  • src/lib/run-with-core.ts
  • src/utils/debug.spec.ts
  • src/utils/debug.ts
  • src/utils/environment.spec.ts
  • src/utils/environment.ts
  • src/utils/exit-codes.spec.ts
  • src/utils/exit-codes.ts
  • src/utils/help-json.spec.ts
  • src/utils/help-json.ts
  • src/utils/interaction-mode.spec.ts
  • src/utils/interaction-mode.ts
  • src/utils/mode-compatibility.spec.ts
  • src/utils/output.spec.ts
  • src/utils/output.ts
  • src/utils/recovery-hints.spec.ts
  • src/utils/recovery-hints.ts
✅ Files skipped from review due to trivial changes (2)
  • src/utils/interaction-mode.spec.ts
  • src/utils/recovery-hints.spec.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/host-probe.spec.ts

Comment thread src/commands/env.ts
: isCiMode()
? 'Name and API key required in CI mode.'
: 'Name and API key required when prompting is unavailable.',
recovery: missingArgsRecovery(undefined, 'Provide environment name and API key as positional arguments.'),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Fix blocking type error in missingArgsRecovery call

Line 46 passes undefined where a string is required, which is breaking typecheck in CI.

Suggested patch
   } else if (!isPromptAllowed()) {
+    const recoveryCommand = formatWorkOSCommand('env add <name> <api-key> --client-id <client-id>');
     exitWithError({
       code: 'missing_args',
       message: isAgentMode()
         ? `Name and API key required in agent mode. Example: ${formatWorkOSCommand('env add staging sk_test_xxx --client-id client_xxx')}`
         : isCiMode()
           ? 'Name and API key required in CI mode.'
           : 'Name and API key required when prompting is unavailable.',
-      recovery: missingArgsRecovery(undefined, 'Provide environment name and API key as positional arguments.'),
+      recovery: missingArgsRecovery(recoveryCommand, 'Provide environment name and API key as positional arguments.'),
     });
   } else {
🧰 Tools
🪛 GitHub Actions: CI / 0_Node 24.txt

[error] 46-46: TypeScript (tsc --noEmit) failed: TS2345 - Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Actions: CI / 1_Node 20.txt

[error] 46-46: TypeScript (tsc) typecheck failed: TS2345: Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Actions: CI / 2_Node 22.txt

[error] 46-46: TypeScript (tsc) error TS2345: Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Actions: CI / Node 20

[error] 46-46: TypeScript typecheck failed (pnpm typecheck). TS2345: Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Actions: CI / Node 22

[error] 46-46: TypeScript (tsc) error TS2345: Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Actions: CI / Node 24

[error] 46-46: TypeScript typecheck failed (tsc --noEmit). TS2345: Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Check: Node 20

[failure] 46-46:
Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Check: Node 22

[failure] 46-46:
Argument of type 'undefined' is not assignable to parameter of type 'string'.

🪛 GitHub Check: Node 24

[failure] 46-46:
Argument of type 'undefined' is not assignable to parameter of type 'string'.

Comment thread src/lib/config-store.ts
Comment on lines 77 to +85
try {
const content = fs.readFileSync(getConfigFilePath(), 'utf-8');
const content = fs.readFileSync(filePath, 'utf-8');
return JSON.parse(content);
} catch (error) {
observeHostFailure('home-fs', error, {
operation: 'read',
target: filePath,
label: 'config fallback file',
});
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Do not classify JSON parse failures as host capability failures

Line 81 and Line 139 currently report home-fs/keychain host failures for any exception, including malformed JSON. That can produce false HOST_EXECUTION_UNTRUSTED diagnostics.

Suggested fix (separate IO/keychain read from JSON parsing)
function readFromFile(): CliConfig | null {
  if (!fileExists()) return null;
  const filePath = getConfigFilePath();
+ let content: string;
  try {
-   const content = fs.readFileSync(filePath, 'utf-8');
-   return JSON.parse(content);
+   content = fs.readFileSync(filePath, 'utf-8');
  } catch (error) {
    observeHostFailure('home-fs', error, {
      operation: 'read',
      target: filePath,
      label: 'config fallback file',
    });
    logWarn('Failed to read config file:', error);
    return null;
  }
+ try {
+   return JSON.parse(content);
+ } catch (error) {
+   logWarn('Failed to parse config file:', error);
+   return null;
+ }
}

function readFromKeyring(): CliConfig | null {
+ let data: string | null;
  try {
    const entry = getKeyringEntry();
-   const data = entry.getPassword();
-   if (!data) return null;
-   return JSON.parse(data);
+   data = entry.getPassword();
  } catch (error) {
    logWarn('Failed to read config from keyring:', error);
    observeHostFailure('keychain', error, {
      operation: 'read',
      target: `${SERVICE_NAME}/${ACCOUNT_NAME}`,
      label: 'config keychain entry',
    });
    return null;
  }
+ if (!data) return null;
+ try {
+   return JSON.parse(data);
+ } catch (error) {
+   logWarn('Failed to parse config from keyring:', error);
+   return null;
+ }
}

Also applies to: 133-143

Comment on lines 55 to +66
function readFromFile(): Credentials | null {
if (!fileExists()) return null;
const filePath = getCredentialsPath();
try {
const content = fs.readFileSync(getCredentialsPath(), 'utf-8');
const content = fs.readFileSync(filePath, 'utf-8');
return JSON.parse(content);
} catch (error) {
observeHostFailure('home-fs', error, {
operation: 'read',
target: filePath,
label: 'credential fallback file',
});
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don’t report JSON parse failures as host capability failures.

The new observeHostFailure(...) calls are inside catch blocks that also capture JSON.parse errors. Corrupt credential payloads are data-integrity issues, not host/sandbox failures, and this can produce false hostExecution trust warnings.

Suggested split of I/O vs parse error handling
 function readFromFile(): Credentials | null {
   if (!fileExists()) return null;
   const filePath = getCredentialsPath();
+  let content: string;
   try {
-    const content = fs.readFileSync(filePath, 'utf-8');
-    return JSON.parse(content);
+    content = fs.readFileSync(filePath, 'utf-8');
   } catch (error) {
     observeHostFailure('home-fs', error, {
       operation: 'read',
       target: filePath,
       label: 'credential fallback file',
     });
     logWarn('Failed to read credentials file:', error);
     return null;
   }
+  try {
+    return JSON.parse(content);
+  } catch (error) {
+    logWarn('Failed to parse credentials file:', error);
+    return null;
+  }
 }
 
 function readFromKeyring(): Credentials | null {
   try {
     const entry = getKeyringEntry();
     const data = entry.getPassword();
     if (!data) {
       logWarn('[credential-store] keyring: entry exists but data is null/empty');
       return null;
     }
-    return JSON.parse(data);
+    try {
+      return JSON.parse(data);
+    } catch (error) {
+      logWarn('[credential-store] keyring payload parse failed:', error);
+      return null;
+    }
   } catch (error) {
     const msg = error instanceof Error ? error.message : String(error);
     logWarn(`[credential-store] keyring read failed: ${msg}`);
     observeHostFailure('keychain', error, {
       operation: 'read',
       target: `${SERVICE_NAME}/${ACCOUNT_NAME}`,
       label: 'credential keychain entry',
     });
     return null;
   }
 }

Also applies to: 112-129

Comment thread src/lib/host-probe.ts
Comment on lines +57 to +63
function isLikelyHostFailure(capability: HostCapability, error: unknown): boolean {
if (capability === 'browser-launch' || capability === 'localhost-bind') {
return true;
}

return isPermissionError(error);
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Unconditional host-failure classification causes false sandbox warnings.

Line 58 currently treats all browser-launch/localhost-bind errors as host-trust failures. This can mislabel ordinary runtime issues (like address-in-use) as sandbox problems.

🐛 Suggested narrowing
 function isLikelyHostFailure(capability: HostCapability, error: unknown): boolean {
-  if (capability === 'browser-launch' || capability === 'localhost-bind') {
-    return true;
-  }
+  const msg = error instanceof Error ? error.message : String(error);
+  if (capability === 'localhost-bind') {
+    if (/\bEADDRINUSE\b|address already in use/i.test(msg)) return false;
+    return isPermissionError(error);
+  }
+  if (capability === 'browser-launch') {
+    return isPermissionError(error) || /\bsandboxd?\b|interaction is not allowed/i.test(msg);
+  }

   return isPermissionError(error);
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
function isLikelyHostFailure(capability: HostCapability, error: unknown): boolean {
if (capability === 'browser-launch' || capability === 'localhost-bind') {
return true;
}
return isPermissionError(error);
}
function isLikelyHostFailure(capability: HostCapability, error: unknown): boolean {
const msg = error instanceof Error ? error.message : String(error);
if (capability === 'localhost-bind') {
if (/\bEADDRINUSE\b|address already in use/i.test(msg)) return false;
return isPermissionError(error);
}
if (capability === 'browser-launch') {
return isPermissionError(error) || /\bsandboxd?\b|interaction is not allowed/i.test(msg);
}
return isPermissionError(error);
}

Comment thread src/lib/run-with-core.ts
Comment thread src/lib/run-with-core.ts
Comment on lines +383 to +386
observeHostFailure('browser-launch', error, {
operation: 'open',
target: deviceAuth.verification_uri_complete,
label: 'installer device auth browser',
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Avoid storing full browser URLs in host-failure details

Line 385 and Line 512 pass full URLs to observeHostFailure. These URLs can include sensitive query params (auth/device codes), which may leak via diagnostics.

Suggested fix (redact query/fragment before reporting)
+function redactUrlForDiagnostics(rawUrl: string): string {
+  try {
+    const u = new URL(rawUrl);
+    return `${u.origin}${u.pathname}`;
+  } catch {
+    return rawUrl;
+  }
+}

 observeHostFailure('browser-launch', error, {
   operation: 'open',
-  target: deviceAuth.verification_uri_complete,
+  target: redactUrlForDiagnostics(deviceAuth.verification_uri_complete),
   label: 'installer device auth browser',
 });

 observeHostFailure('browser-launch', error, {
   operation: 'open',
-  target: inspectUrl,
+  target: redactUrlForDiagnostics(inspectUrl),
   label: 'XState inspector browser',
 });

Also applies to: 510-513

Comment thread src/utils/help-json.ts
Comment on lines +1331 to +1357
const helpJsonCommandNames = new Set([
...commands.map((command) => command.name.split(' ')[0]),
...Object.keys(commandAliases),
]);

/**
* Extract the requested command from raw argv before yargs parses --help.
*
* This intentionally matches only known command names so option values from
* global flags like `--mode agent` are not mistaken for commands.
*/
export function extractHelpJsonCommand(argv: string[]): string | undefined {
for (let i = 0; i < argv.length; i++) {
const arg = argv[i];
if (arg === '--mode') {
i += 1;
continue;
}
if (arg.startsWith('--mode=')) {
continue;
}
if (!arg.startsWith('-') && helpJsonCommandNames.has(arg)) {
return commandAliases[arg] ?? arg;
}
}
return undefined;
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

extractHelpJsonCommand loses multi-token commands (e.g. auth login).

Line 1332 only indexes first tokens, so ['auth','login','--help','--json'] resolves to auth, which misses buildCommandTree() exact matching and returns the full tree.

Suggested fix
-const helpJsonCommandNames = new Set([
-  ...commands.map((command) => command.name.split(' ')[0]),
-  ...Object.keys(commandAliases),
-]);
+const helpJsonCommandNames = new Set(commands.map((command) => command.name));

 export function extractHelpJsonCommand(argv: string[]): string | undefined {
+  const tokens: string[] = [];
   for (let i = 0; i < argv.length; i++) {
     const arg = argv[i];
     if (arg === '--mode') {
       i += 1;
       continue;
     }
     if (arg.startsWith('--mode=')) {
       continue;
     }
-    if (!arg.startsWith('-') && helpJsonCommandNames.has(arg)) {
-      return commandAliases[arg] ?? arg;
+    if (!arg.startsWith('-')) {
+      tokens.push(commandAliases[arg] ?? arg);
     }
   }
+  for (let len = tokens.length; len > 0; len--) {
+    const candidate = tokens.slice(0, len).join(' ');
+    if (helpJsonCommandNames.has(candidate)) return candidate;
+  }
   return undefined;
 }

…covery hints

- resolveEffectiveOutputMode forces JSON when WORKOS_MODE is explicitly
  agent or ci, preventing human-formatted noise in machine streams
- Shell-safe confirmation commands via buildConfirmationCommand with
  proper quoting for --data JSON values
- Omit recovery command when it cannot be deterministically rebuilt
  (stdin body, explicit API key)
- Suppress human warnings (emoji/chalk) in JSON output mode
- Skip update check when prompting is disallowed
Comment thread src/utils/debug.ts
Comment on lines 93 to +100
writeLog('WARN', '⚠️ ', args);
}

export function logVisibleWarn(...args: unknown[]): void {
const msg = writeLog('WARN', '⚠️ ', args);
if (!debugEnabled && !isJsonMode()) {
console.error(chalk.yellow(`⚠️ ${msg}`));
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 logVisibleWarn is always silent in JSON mode — sandbox warning never reaches agents

logVisibleWarn guards its only console.error call with !isJsonMode(), and writeLog also skips its clack output when isJsonMode() is true. So in agent/CI mode with JSON output — the exact context this feature targets — both warnIfSandboxed() and observeHostFailure() silently swallow the warning. An agent running inside a sandbox receives auth_required errors with no indication that the host filesystem or keychain is unreachable. The warning should be emitted to stderr as a structured JSON object in JSON mode so agents can parse it.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/commands/api/index.spec.ts (1)

2-4: ⚡ Quick win

Replace sync fs APIs with async alternatives in test setup and cleanup.

The test functions are already async, so mkdtempSync, writeFileSync, and rmSync (lines 2, 417, 419, 436) can be replaced with their async equivalents from fs/promises without changing test behavior.

Proposed refactor
-import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { mkdtemp, rm, writeFile } from 'node:fs/promises';
@@
-    const dir = mkdtempSync(join(tmpdir(), 'workos-api-'));
+    const dir = await mkdtemp(join(tmpdir(), 'workos-api-'));
@@
-    writeFileSync(file, '{"name":"Acme"}');
+    await writeFile(file, '{"name":"Acme"}');
@@
-      rmSync(dir, { recursive: true, force: true });
+      await rm(dir, { recursive: true, force: true });

Per coding guidelines: src/**/*.{ts,tsx,js}: Avoid Node-specific sync APIs (crypto, fs sync) unless necessary


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d45bdf39-6e34-4440-a274-20ab8a82bb7d

📥 Commits

Reviewing files that changed from the base of the PR and between e2210d8 and d174758.

📒 Files selected for processing (13)
  • src/bin.ts
  • src/commands/api/index.spec.ts
  • src/commands/api/index.ts
  • src/commands/env.spec.ts
  • src/utils/command-invocation.spec.ts
  • src/utils/command-invocation.ts
  • src/utils/debug.spec.ts
  • src/utils/debug.ts
  • src/utils/mode-compatibility.spec.ts
  • src/utils/output.spec.ts
  • src/utils/output.ts
  • src/utils/recovery-hints.spec.ts
  • src/utils/recovery-hints.ts
🚧 Files skipped from review as they are similar to previous changes (6)
  • src/utils/recovery-hints.spec.ts
  • src/commands/env.spec.ts
  • src/utils/recovery-hints.ts
  • src/commands/api/index.ts
  • src/bin.ts
  • src/utils/debug.ts

@nicknisi nicknisi merged commit 4a6e8cc into main May 12, 2026
6 checks passed
@nicknisi nicknisi deleted the nick/host-capability-probe branch May 12, 2026 19:51
nicknisi added a commit to workos/skills that referenced this pull request May 13, 2026
* feat(workos): add CLI agent-mode guidance to skill

Coding agents now have explicit instructions for invoking the WorkOS CLI:

- Recommend `WORKOS_MODE=agent workos doctor --json --skip-ai` as the
  setup/debugging preflight.
- Document the two-axis model: `--json` is output formatting only;
  `WORKOS_MODE=agent` controls prompts, browser fallback, and host trust.
- Treat `HOST_EXECUTION_UNTRUSTED` from doctor as a hard sandbox trust
  boundary that requires re-running on the host shell before trusting
  auth/config/API failures.
- Note legacy compatibility for `WORKOS_NO_PROMPT` and `WORKOS_FORCE_TTY`.
- Update workos-widgets install prompts to use `WORKOS_MODE=agent` when
  asking the user to run `npx workos@latest install`.

The CLI surface (mode resolver, doctor visibility, recovery hints) is
landing in the workos/cli repo (workos/cli#143);
this skill update teaches agents to use it.

* fix(workos): tighten agent-mode guidance for precision and rot-resistance

Reframe guardrail as prohibition, explain --skip-ai flag, add --help
fallback for confirmation flags, clarify WORKOS_NO_PROMPT migration
path, and remove redundant parenthetical from widgets workflow.

* fix(workos): address second-pass review findings

- Fix broken WORKOS_NO_PROMPT migration syntax (was not a runnable fragment)
- Add CLI/agent-mode/doctor to skill description for triggering
- Drop "machine-readable" framing from widgets to avoid implying agent
  mode replaces --json
- Add --skip-ai version fallback to cli-upgrade reference
- Soften doctor JSON shape claims to resist schema rot

* fix(workos): align agent-mode guidance with shipped CLI implementation

- Soften guardrail from "never run without" to "prefer" since the CLI
  auto-detects CLAUDECODE, CLAUDE_CODE, CURSOR_AGENT, CODEX_SANDBOX,
  CURSOR_TRACE_ID, and non-TTY
- Document the two-axis model (interaction mode vs output mode) with
  the three possible interaction values (human/agent/ci)
- Restore precise doctor field names (interactionMode, hostExecution)
  since they're stable type definitions in the CLI
- Fix recovery hints shape: describe per-hint fields (description,
  command, hostShellRequired) and note JSON is emitted on stderr
- Enumerate all known confirmation flags including debug reset
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant