Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,8 @@ Test definitions are now type-checked by `flutter analyze` — a misspelt step n

**v0.9.7** adds **biometric authentication testing** — `enroll biometric`, `biometric match`, `biometric no match` steps (and matching `EnrollBiometric()` / `BiometricMatch()` / `BiometricNoMatch()` annotation classes) drive Face ID / Touch ID / fingerprint flows on iOS Simulator and Android emulator. Skipped on physical devices.

**v0.9.8** fixes biometric no-match on **iOS 26+ simulator** where `notifyutil` no-match notifications no longer resolve `LAContext.evaluatePolicy`. The CLI now delivers results via `probe.biometric_signal`; use `awaitBiometricResult()` from `flutter_probe_agent` instead of `local_auth.authenticate()` in PROBE_AGENT builds. Also adds a **port-range fallback**: the agent auto-tries ports 48686–48695 and logs `PROBE_PORT_BUSY=N (another probe agent is running)` when a collision is with a sibling agent.

Full reference: [flutterprobe.dev/probescript/annotations](https://flutterprobe.dev/probescript/annotations/) (or [`docs/wiki/Annotations.md`](docs/wiki/Annotations.md) on GitHub).

## CLI Commands
Expand Down
21 changes: 21 additions & 0 deletions probe_agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,26 @@ xcrun devicectl device process launch --device <UDID> <bundle-id>
probe test tests/ --host <device-ip> --token <probe-token> -v
```

## Biometric Authentication Testing (v0.9.7+)

`enroll biometric`, `biometric match`, and `biometric no match` ProbeScript steps drive Face ID / Touch ID / fingerprint flows on iOS Simulator and Android emulator. On iOS 26+ simulator, the notifyutil `no-match` notification no longer resolves `LAContext.evaluatePolicy`. Use `awaitBiometricResult()` in your screen widget to receive the result from the CLI via a Dart `Completer`:

```dart
import 'package:flutter_probe_agent/flutter_probe_agent.dart';
import 'package:local_auth/local_auth.dart';

Future<bool> _signIn() async {
if (const bool.fromEnvironment('PROBE_AGENT')) {
// CLI delivers true (match) or false (no-match) via probe.biometric_signal.
// Works on all iOS simulator versions including iOS 26+.
return awaitBiometricResult();
}
return LocalAuthentication().authenticate(localizedReason: 'Sign in');
}
```

The CLI automatically sends `probe.biometric_signal` after every `biometric match` / `biometric no match` step — no changes to `.probe` test files are needed.

## Features

- **WebSocket + HTTP transports** — persistent connection for simulators, stateless HTTP for physical devices
Expand All @@ -123,6 +143,7 @@ probe test tests/ --host <device-ip> --token <probe-token> -v
- **WiFi testing** — bind to `0.0.0.0` with `PROBE_WIFI=true` for cable-free testing
- **Pre-shared restart token** — `restart the app` works over WiFi without USB log reading
- **`tap "X" if visible`** — conditional actions that skip silently when widget is not found
- **Port-range fallback** — auto-tries ports 48686–48695 if preferred port is busy; logs `PROBE_PORT_BUSY=N (another probe agent is running)` when collision is with a sibling agent

## Requirements

Expand Down
45 changes: 45 additions & 0 deletions website/src/content/docs/platform/ios.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,51 @@ if (!probeEnabled) {
Build with `--dart-define=PROBE_AGENT=true` to skip these requests during testing.
:::

## Biometric Authentication (Face ID / Touch ID)

`enroll biometric`, `biometric match`, and `biometric no match` drive Face ID and Touch ID flows on iOS Simulator via `xcrun simctl spawn booted notifyutil`.

```
before all tests
enroll biometric

test "Face ID unlocks the app"
open the app
tap "Sign in with Face ID"
wait until "Sign in with Face ID" appears
tap "Sign in with Face ID"
biometric match
wait until "Dashboard" appears
see "Dashboard"

test "failed Face ID shows error"
open the app
tap "Sign in with Face ID"
wait until "Sign in with Face ID" appears
tap "Sign in with Face ID"
biometric no match
wait until "Authentication failed" appears
see "Authentication failed"
```

:::caution[iOS 26+ simulator — use `awaitBiometricResult()` in your app]
On iOS 26 / Xcode 26.5, the `faceCapture.no-match` notifyutil notification **no longer resolves `LAContext.evaluatePolicy`**. If your app calls `local_auth.authenticate()` directly, no-match tests will hang indefinitely.

**Fix:** In PROBE_AGENT builds, use `awaitBiometricResult()` from `flutter_probe_agent` instead. The CLI delivers the result via `probe.biometric_signal` after every `biometric match` / `biometric no match` step:

```dart
import 'package:flutter_probe_agent/flutter_probe_agent.dart';

final ok = const bool.fromEnvironment('PROBE_AGENT')
? await awaitBiometricResult()
: await LocalAuthentication().authenticate(localizedReason: 'Sign in');
```

This pattern works on all iOS versions and requires no changes to `.probe` test files.
:::

**Multiple simulators:** If more than one simulator is booted, the biometric notifyutil fires on `booted` (which selects an arbitrary device). Always boot only the target simulator, or pass `--device <UDID>` so the CLI targets the right one.

## Video Recording

iOS simulator video is recorded via `xcrun simctl io <UDID> recordVideo`. Videos use the `h264` codec (not HEVC) for browser compatibility in HTML reports.
Expand Down
24 changes: 18 additions & 6 deletions website/src/content/docs/probescript/annotations.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,12 +238,14 @@ Both workflows are valid:

Pick whichever fits your team.

## Biometric authentication (v0.9.7+)
## Biometric authentication (v0.9.8+)

Test Face ID, Touch ID, and Android fingerprint flows end-to-end without
real hardware. FlutterProbe drives the simulator/emulator's biometric
controls via `xcrun simctl spawn notifyutil` (iOS) and
`adb -s <serial> emu finger touch` (Android).
real hardware. The CLI fires platform-level biometric commands and then
delivers the result to the agent via `probe.biometric_signal` so apps can
use `awaitBiometricResult()` instead of `local_auth.authenticate()` — required
on iOS 26+ simulator where `notifyutil` no-match notifications no longer
resolve `LAContext.evaluatePolicy`.

```dart
@ProbeSuite(
Expand All @@ -270,8 +272,18 @@ class BiometricAuthScreen {}
| Step | iOS Simulator | Android emulator |
|---|---|---|
| `EnrollBiometric()` | Posts `com.apple.BiometricKit.enrollmentChanged` Darwin notification | No-op (pre-enroll fingerprint ID 1 in Settings) |
| `BiometricMatch()` | Posts `*_Sim.faceCapture.match` + `*_Sim.fingerTouch.match` | `adb emu finger touch 1` |
| `BiometricNoMatch()` | Posts `*_Sim.faceCapture.no-match` + `*_Sim.fingerTouch.no-match` | `adb emu finger touch 9999` (unregistered id) |
| `BiometricMatch()` | Posts `*_Sim.faceCapture.match` + sends `probe.biometric_signal {result: true}` | `adb emu finger touch 1` + signal |
| `BiometricNoMatch()` | Posts `*_Sim.faceCapture.no-match` + sends `probe.biometric_signal {result: false}` | `adb emu finger touch 9999` + signal |

After each biometric step the CLI sends `probe.biometric_signal` to the agent so the result is always delivered reliably — regardless of iOS version. Your screen widget should use `awaitBiometricResult()` from `flutter_probe_agent` in PROBE_AGENT builds:

```dart
import 'package:flutter_probe_agent/flutter_probe_agent.dart';

final ok = const bool.fromEnvironment('PROBE_AGENT')
? await awaitBiometricResult() // resolved by CLI via probe.biometric_signal
: await localAuth.authenticate(...); // production path
```

**Physical devices are skipped with a warning.** Real Face ID / Touch ID
requires an actual face or finger and can't be programmatically driven —
Expand Down
15 changes: 11 additions & 4 deletions website/src/content/docs/probescript/syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,12 +199,19 @@ test "non-matching face is rejected"
```

On iOS, this posts the `BiometricKit_Sim.faceCapture.match` / `.no-match`
Darwin notifications (and the `fingerTouch.*` equivalents, so the same
step works on Touch ID devices). On Android, this calls
`adb -s <serial> emu finger touch <id>` — fingerprint ID `1` is
matching by convention (must be pre-enrolled in Settings before tests
Darwin notifications (and the `fingerTouch.*` equivalents for Touch ID
devices), then sends `probe.biometric_signal` to the agent. On Android,
this calls `adb -s <serial> emu finger touch <id>` — fingerprint ID `1`
is matching by convention (must be pre-enrolled in Settings before tests
run); any unregistered ID is no-match.

:::caution[iOS 26+ — use `awaitBiometricResult()` in your app]
On iOS 26+ simulator the `no-match` notification no longer resolves
`LAContext.evaluatePolicy`. Use `awaitBiometricResult()` from
`flutter_probe_agent` in PROBE_AGENT builds — the CLI resolves it via
`probe.biometric_signal`. See the [iOS platform guide](/platform/ios/#biometric-authentication-face-id--touch-id) for the code pattern.
:::

## HTTP Calls

Make real HTTP requests to APIs (runs on the CLI, not the device):
Expand Down
2 changes: 1 addition & 1 deletion website/src/content/docs/tools/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | probe-mcp
Expected response:

```json
{"jsonrpc":"2.0","id":1,"result":{"capabilities":{"tools":{}},"protocolVersion":"2024-11-05","serverInfo":{"name":"probe-mcp","version":"0.9.7"}}}
{"jsonrpc":"2.0","id":1,"result":{"capabilities":{"tools":{}},"protocolVersion":"2024-11-05","serverInfo":{"name":"probe-mcp","version":"0.9.8"}}}
```

List all available tools:
Expand Down
Loading