From d29081da932eca261459b273d4ed358f1ff01054 Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Tue, 12 May 2026 23:30:59 -0600 Subject: [PATCH] docs: update biometric docs for iOS 26+ / v0.9.8 - probe_agent/README.md: document awaitBiometricResult() pattern + port-range feature - platform/ios.md: add Face ID / biometric section with iOS 26+ caution - annotations.md: correct biometric table (probe.biometric_signal, not just notifyutil); add awaitBiometricResult() code example - syntax.md: add iOS 26+ caution block linking to ios.md - README.md: add v0.9.8 entry for biometric signal fix and port-range fallback - mcp.md: bump version to 0.9.8 --- README.md | 2 + probe_agent/README.md | 21 +++++++++ website/src/content/docs/platform/ios.md | 45 +++++++++++++++++++ .../content/docs/probescript/annotations.md | 24 +++++++--- .../src/content/docs/probescript/syntax.md | 15 +++++-- website/src/content/docs/tools/mcp.md | 2 +- 6 files changed, 98 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index e6b6159..1ea7b24 100644 --- a/README.md +++ b/README.md @@ -419,6 +419,8 @@ Test definitions are now type-checked by `flutter analyze` — a misspelt step n **v0.9.7** adds **biometric authentication testing** — `enroll biometric`, `biometric match`, `biometric no match` steps (and matching `EnrollBiometric()` / `BiometricMatch()` / `BiometricNoMatch()` annotation classes) drive Face ID / Touch ID / fingerprint flows on iOS Simulator and Android emulator. Skipped on physical devices. +**v0.9.8** fixes biometric no-match on **iOS 26+ simulator** where `notifyutil` no-match notifications no longer resolve `LAContext.evaluatePolicy`. The CLI now delivers results via `probe.biometric_signal`; use `awaitBiometricResult()` from `flutter_probe_agent` instead of `local_auth.authenticate()` in PROBE_AGENT builds. Also adds a **port-range fallback**: the agent auto-tries ports 48686–48695 and logs `PROBE_PORT_BUSY=N (another probe agent is running)` when a collision is with a sibling agent. + Full reference: [flutterprobe.dev/probescript/annotations](https://flutterprobe.dev/probescript/annotations/) (or [`docs/wiki/Annotations.md`](docs/wiki/Annotations.md) on GitHub). ## CLI Commands diff --git a/probe_agent/README.md b/probe_agent/README.md index 496ee51..45c4327 100644 --- a/probe_agent/README.md +++ b/probe_agent/README.md @@ -115,6 +115,26 @@ xcrun devicectl device process launch --device probe test tests/ --host --token -v ``` +## Biometric Authentication Testing (v0.9.7+) + +`enroll biometric`, `biometric match`, and `biometric no match` ProbeScript steps drive Face ID / Touch ID / fingerprint flows on iOS Simulator and Android emulator. On iOS 26+ simulator, the notifyutil `no-match` notification no longer resolves `LAContext.evaluatePolicy`. Use `awaitBiometricResult()` in your screen widget to receive the result from the CLI via a Dart `Completer`: + +```dart +import 'package:flutter_probe_agent/flutter_probe_agent.dart'; +import 'package:local_auth/local_auth.dart'; + +Future _signIn() async { + if (const bool.fromEnvironment('PROBE_AGENT')) { + // CLI delivers true (match) or false (no-match) via probe.biometric_signal. + // Works on all iOS simulator versions including iOS 26+. + return awaitBiometricResult(); + } + return LocalAuthentication().authenticate(localizedReason: 'Sign in'); +} +``` + +The CLI automatically sends `probe.biometric_signal` after every `biometric match` / `biometric no match` step — no changes to `.probe` test files are needed. + ## Features - **WebSocket + HTTP transports** — persistent connection for simulators, stateless HTTP for physical devices @@ -123,6 +143,7 @@ probe test tests/ --host --token -v - **WiFi testing** — bind to `0.0.0.0` with `PROBE_WIFI=true` for cable-free testing - **Pre-shared restart token** — `restart the app` works over WiFi without USB log reading - **`tap "X" if visible`** — conditional actions that skip silently when widget is not found +- **Port-range fallback** — auto-tries ports 48686–48695 if preferred port is busy; logs `PROBE_PORT_BUSY=N (another probe agent is running)` when collision is with a sibling agent ## Requirements diff --git a/website/src/content/docs/platform/ios.md b/website/src/content/docs/platform/ios.md index c62ea5d..329dcca 100644 --- a/website/src/content/docs/platform/ios.md +++ b/website/src/content/docs/platform/ios.md @@ -114,6 +114,51 @@ if (!probeEnabled) { Build with `--dart-define=PROBE_AGENT=true` to skip these requests during testing. ::: +## Biometric Authentication (Face ID / Touch ID) + +`enroll biometric`, `biometric match`, and `biometric no match` drive Face ID and Touch ID flows on iOS Simulator via `xcrun simctl spawn booted notifyutil`. + +``` +before all tests + enroll biometric + +test "Face ID unlocks the app" + open the app + tap "Sign in with Face ID" + wait until "Sign in with Face ID" appears + tap "Sign in with Face ID" + biometric match + wait until "Dashboard" appears + see "Dashboard" + +test "failed Face ID shows error" + open the app + tap "Sign in with Face ID" + wait until "Sign in with Face ID" appears + tap "Sign in with Face ID" + biometric no match + wait until "Authentication failed" appears + see "Authentication failed" +``` + +:::caution[iOS 26+ simulator — use `awaitBiometricResult()` in your app] +On iOS 26 / Xcode 26.5, the `faceCapture.no-match` notifyutil notification **no longer resolves `LAContext.evaluatePolicy`**. If your app calls `local_auth.authenticate()` directly, no-match tests will hang indefinitely. + +**Fix:** In PROBE_AGENT builds, use `awaitBiometricResult()` from `flutter_probe_agent` instead. The CLI delivers the result via `probe.biometric_signal` after every `biometric match` / `biometric no match` step: + +```dart +import 'package:flutter_probe_agent/flutter_probe_agent.dart'; + +final ok = const bool.fromEnvironment('PROBE_AGENT') + ? await awaitBiometricResult() + : await LocalAuthentication().authenticate(localizedReason: 'Sign in'); +``` + +This pattern works on all iOS versions and requires no changes to `.probe` test files. +::: + +**Multiple simulators:** If more than one simulator is booted, the biometric notifyutil fires on `booted` (which selects an arbitrary device). Always boot only the target simulator, or pass `--device ` so the CLI targets the right one. + ## Video Recording iOS simulator video is recorded via `xcrun simctl io recordVideo`. Videos use the `h264` codec (not HEVC) for browser compatibility in HTML reports. diff --git a/website/src/content/docs/probescript/annotations.md b/website/src/content/docs/probescript/annotations.md index cfcf9d3..f25a512 100644 --- a/website/src/content/docs/probescript/annotations.md +++ b/website/src/content/docs/probescript/annotations.md @@ -238,12 +238,14 @@ Both workflows are valid: Pick whichever fits your team. -## Biometric authentication (v0.9.7+) +## Biometric authentication (v0.9.8+) Test Face ID, Touch ID, and Android fingerprint flows end-to-end without -real hardware. FlutterProbe drives the simulator/emulator's biometric -controls via `xcrun simctl spawn notifyutil` (iOS) and -`adb -s emu finger touch` (Android). +real hardware. The CLI fires platform-level biometric commands and then +delivers the result to the agent via `probe.biometric_signal` so apps can +use `awaitBiometricResult()` instead of `local_auth.authenticate()` — required +on iOS 26+ simulator where `notifyutil` no-match notifications no longer +resolve `LAContext.evaluatePolicy`. ```dart @ProbeSuite( @@ -270,8 +272,18 @@ class BiometricAuthScreen {} | Step | iOS Simulator | Android emulator | |---|---|---| | `EnrollBiometric()` | Posts `com.apple.BiometricKit.enrollmentChanged` Darwin notification | No-op (pre-enroll fingerprint ID 1 in Settings) | -| `BiometricMatch()` | Posts `*_Sim.faceCapture.match` + `*_Sim.fingerTouch.match` | `adb emu finger touch 1` | -| `BiometricNoMatch()` | Posts `*_Sim.faceCapture.no-match` + `*_Sim.fingerTouch.no-match` | `adb emu finger touch 9999` (unregistered id) | +| `BiometricMatch()` | Posts `*_Sim.faceCapture.match` + sends `probe.biometric_signal {result: true}` | `adb emu finger touch 1` + signal | +| `BiometricNoMatch()` | Posts `*_Sim.faceCapture.no-match` + sends `probe.biometric_signal {result: false}` | `adb emu finger touch 9999` + signal | + +After each biometric step the CLI sends `probe.biometric_signal` to the agent so the result is always delivered reliably — regardless of iOS version. Your screen widget should use `awaitBiometricResult()` from `flutter_probe_agent` in PROBE_AGENT builds: + +```dart +import 'package:flutter_probe_agent/flutter_probe_agent.dart'; + +final ok = const bool.fromEnvironment('PROBE_AGENT') + ? await awaitBiometricResult() // resolved by CLI via probe.biometric_signal + : await localAuth.authenticate(...); // production path +``` **Physical devices are skipped with a warning.** Real Face ID / Touch ID requires an actual face or finger and can't be programmatically driven — diff --git a/website/src/content/docs/probescript/syntax.md b/website/src/content/docs/probescript/syntax.md index 342613f..e3bcd16 100644 --- a/website/src/content/docs/probescript/syntax.md +++ b/website/src/content/docs/probescript/syntax.md @@ -199,12 +199,19 @@ test "non-matching face is rejected" ``` On iOS, this posts the `BiometricKit_Sim.faceCapture.match` / `.no-match` -Darwin notifications (and the `fingerTouch.*` equivalents, so the same -step works on Touch ID devices). On Android, this calls -`adb -s emu finger touch ` — fingerprint ID `1` is -matching by convention (must be pre-enrolled in Settings before tests +Darwin notifications (and the `fingerTouch.*` equivalents for Touch ID +devices), then sends `probe.biometric_signal` to the agent. On Android, +this calls `adb -s emu finger touch ` — fingerprint ID `1` +is matching by convention (must be pre-enrolled in Settings before tests run); any unregistered ID is no-match. +:::caution[iOS 26+ — use `awaitBiometricResult()` in your app] +On iOS 26+ simulator the `no-match` notification no longer resolves +`LAContext.evaluatePolicy`. Use `awaitBiometricResult()` from +`flutter_probe_agent` in PROBE_AGENT builds — the CLI resolves it via +`probe.biometric_signal`. See the [iOS platform guide](/platform/ios/#biometric-authentication-face-id--touch-id) for the code pattern. +::: + ## HTTP Calls Make real HTTP requests to APIs (runs on the CLI, not the device): diff --git a/website/src/content/docs/tools/mcp.md b/website/src/content/docs/tools/mcp.md index 1a90f20..471b092 100644 --- a/website/src/content/docs/tools/mcp.md +++ b/website/src/content/docs/tools/mcp.md @@ -238,7 +238,7 @@ echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | probe-mcp Expected response: ```json -{"jsonrpc":"2.0","id":1,"result":{"capabilities":{"tools":{}},"protocolVersion":"2024-11-05","serverInfo":{"name":"probe-mcp","version":"0.9.7"}}} +{"jsonrpc":"2.0","id":1,"result":{"capabilities":{"tools":{}},"protocolVersion":"2024-11-05","serverInfo":{"name":"probe-mcp","version":"0.9.8"}}} ``` List all available tools: