From 579cbceb27b58c43ebf930ba731cd2e0d8840c79 Mon Sep 17 00:00:00 2001 From: Matthew Mellor Date: Mon, 4 May 2026 20:39:10 -0500 Subject: [PATCH 1/2] feat(makefile): plugin execution loop and JSON aggregation (Story 13.5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wires loaded plugins into the existing _lint/_format/_fix/_test/_security recipes via a new lib/plugin-execute.sh. Each recipe sources the lib at its top, then calls dispatch_plugin_target after the last HAS_ block; plugin results aggregate into the same ran_languages / failed_languages / JSON envelope as core results. What ships: - lib/plugin-execute.sh — sourceable helpers (evaluate_gate, render_cmd, apply_override, dispatch_plugin_target). Reads from the loader cache populated by Story 13.2; uses log_event from lib/log.sh; runs cmds via bash -c (no eval). - Per-target gate evaluation (file / dir / glob match; absolute paths rejected with structured error). Empty / missing gate = always run. - {paths} interpolation from paths_var, filtered to existing paths (mirrors the RUBY_PATHS pattern in core). - Per-language overrides for plugin languages: .devrail.yml entries like `elixir: { linter: dialyxir }` replace targets..cmd. Map: lint→linter, format_check/format_fix→formatter, fix→fixer, test→test, security→security. - DEVRAIL_FAIL_FAST=1 short-circuits the dispatcher and the recipe. - No-op when plugins: is absent — v1.9.x consumers see byte-identical JSON output. - SHELL := /bin/bash so recipes can source bash-only libs (plugin- execute.sh uses [[, ((, indirect parameter expansion). Existing POSIX recipes remain valid. - tests/test-plugin-execution.sh — 10 cases exercising no-op, pass, fail, gate skip / run, paths interpolation, override, fail-fast, partial targets, JSON regression. New CI step "Plugin execution smoke test" added to .github/workflows/ci.yml. Closes Story 13.5 (Epic 13). Final implementation story before the v1.10.0 marketing release (Story 13.6). Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/ci.yml | 11 + CHANGELOG.md | 12 ++ Makefile | 61 +++++- STABILITY.md | 2 +- lib/plugin-execute.sh | 256 ++++++++++++++++++++++ tests/test-plugin-execution.sh | 375 +++++++++++++++++++++++++++++++++ 6 files changed, 711 insertions(+), 6 deletions(-) create mode 100644 lib/plugin-execute.sh create mode 100755 tests/test-plugin-execution.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index dc1b3ea..8affa6d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -83,6 +83,17 @@ jobs: DEVRAIL_IMAGE: ${{ env.IMAGE_NAME }} DEVRAIL_TAG: ${{ env.IMAGE_TAG }} + # Phase 2g: Plugin execution smoke test (Story 13.5) + # Drives lib/plugin-execute.sh against hand-crafted loader-cache + # fixtures: dispatcher no-op, pass / fail, gate skip / run, paths + # interpolation, per-language override, DEVRAIL_FAIL_FAST short-circuit, + # partial targets, and JSON-shape regression. + - name: Plugin execution smoke test + run: bash tests/test-plugin-execution.sh + env: + DEVRAIL_IMAGE: ${{ env.IMAGE_NAME }} + DEVRAIL_TAG: ${{ env.IMAGE_TAG }} + # Phase 3: Security scans # Blocking scan: OS packages only. We control the base image and can act on # these. ignore-unfixed skips CVEs with no Debian patch available yet. diff --git a/CHANGELOG.md b/CHANGELOG.md index e5a7af5..4610c46 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- Plugin execution loop and JSON aggregation (Story 13.5, Epic 13 / v1.10.x preview): + - **`_lint` / `_format` / `_fix` / `_test` / `_security` now dispatch each loaded plugin's matching target after the core `HAS_` blocks.** The new `lib/plugin-execute.sh` library exposes `dispatch_plugin_target` (per-recipe), `evaluate_gate`, `render_cmd`, and `apply_override`. Plugin results enter the existing `ran_languages` / `failed_languages` arrays and the same JSON envelope — consumers cannot distinguish plugin vs core results from the JSON shape. + - **Per-target gate evaluation.** When a plugin manifest declares `gates: { : [path, ...] }`, the dispatcher only runs the target when every gate path exists (file, directory, or glob match). Absolute paths are rejected. Skips emit a structured `plugin gate skipped` event. + - **`{paths}` interpolation.** When a target's `cmd` references `{paths}`, the value of `${}` (filtered to existing paths) is substituted, mirroring how `RUBY_PATHS` is filtered in the core Ruby block. Falls back to `paths_default` when the env var is unset. + - **Per-language overrides for plugin languages.** `.devrail.yml` entries like `elixir: { linter: dialyxir }` replace the manifest's default `targets..cmd` for that target. Override key map: `lint` → `linter`, `format_check` / `format_fix` → `formatter`, `fix` → `fixer`, `test` → `test`, `security` → `security`. + - **`DEVRAIL_FAIL_FAST=1` parity.** A plugin failure under fail-fast short-circuits the dispatcher and the recipe — no later plugins run. + - **No-op when `plugins:` is absent.** `_plugins-load` writes an empty cache; the dispatcher exits immediately. v1.9.x consumers see byte-identical JSON output and no behavioural change. + - **`SHELL := /bin/bash`** — the Makefile now pins recipes to bash so `lib/plugin-execute.sh` (uses `[[`, `((`, indirect parameter expansion) can be sourced directly. Existing POSIX-sh recipes remain valid. + - Smoke test: `tests/test-plugin-execution.sh` exercises 10 cases (no-op, pass, fail, gate skip / run, paths interpolation, override, fail-fast, partial targets, JSON regression). + ## [1.10.4] - 2026-05-04 ### Added diff --git a/Makefile b/Makefile index 59a3fc6..e21084d 100644 --- a/Makefile +++ b/Makefile @@ -16,6 +16,11 @@ # --------------------------------------------------------------------------- # Variables (overridable via environment) # --------------------------------------------------------------------------- +# Use bash for all recipes so we can source bash-only libs (plugin-execute.sh +# uses [[, ((, and indirect parameter expansion). Existing POSIX-sh recipes +# remain valid because bash is a superset. +SHELL := /bin/bash + DEVRAIL_IMAGE ?= ghcr.io/devrail-dev/dev-toolchain DEVRAIL_TAG ?= local DEVRAIL_FAIL_FAST ?= 0 @@ -365,8 +370,15 @@ _plugins-load: _plugins-verify if [ "$$failed" -gt 0 ]; then exit 2; fi # --- _lint: language-specific linting --- +# After the core HAS_ blocks, plugin targets (Story 13.5) are dispatched +# via lib/plugin-execute.sh. The dispatcher updates overall_exit / +# ran_languages / failed_languages in this recipe's shell scope, so plugin +# results aggregate into the same JSON envelope as core results. The +# DEVRAIL_FAIL_FAST short-circuit pattern is mirrored after the dispatch +# call to keep behaviour symmetric with the per-language blocks. _lint: _plugins-load - @start_time=$$(date +%s%3N); \ + @. /opt/devrail/lib/plugin-execute.sh; \ + start_time=$$(date +%s%3N); \ overall_exit=0; \ ran_languages=""; \ failed_languages=""; \ @@ -572,6 +584,13 @@ _lint: _plugins-load exit $$overall_exit; \ fi; \ fi; \ + dispatch_plugin_target lint; \ + if [ "$(DEVRAIL_FAIL_FAST)" = "1" ] && [ $$overall_exit -ne 0 ]; then \ + end_time=$$(date +%s%3N); \ + duration=$$((end_time - start_time)); \ + echo "{\"target\":\"lint\",\"status\":\"fail\",\"duration_ms\":$$duration,\"languages\":[$${ran_languages%,}],\"failed\":[$${failed_languages%,}]}"; \ + exit $$overall_exit; \ + fi; \ end_time=$$(date +%s%3N); \ duration=$$((end_time - start_time)); \ if [ $$overall_exit -eq 0 ]; then \ @@ -583,7 +602,8 @@ _lint: _plugins-load # --- _format: language-specific format checking --- _format: _plugins-load - @start_time=$$(date +%s%3N); \ + @. /opt/devrail/lib/plugin-execute.sh; \ + start_time=$$(date +%s%3N); \ overall_exit=0; \ ran_languages=""; \ failed_languages=""; \ @@ -722,6 +742,13 @@ _format: _plugins-load exit $$overall_exit; \ fi; \ fi; \ + dispatch_plugin_target format_check; \ + if [ "$(DEVRAIL_FAIL_FAST)" = "1" ] && [ $$overall_exit -ne 0 ]; then \ + end_time=$$(date +%s%3N); \ + duration=$$((end_time - start_time)); \ + echo "{\"target\":\"format\",\"status\":\"fail\",\"duration_ms\":$$duration,\"languages\":[$${ran_languages%,}],\"failed\":[$${failed_languages%,}]}"; \ + exit $$overall_exit; \ + fi; \ end_time=$$(date +%s%3N); \ duration=$$((end_time - start_time)); \ if [ $$overall_exit -eq 0 ]; then \ @@ -733,7 +760,8 @@ _format: _plugins-load # --- _fix: language-specific format fixing (in-place) --- _fix: _plugins-load - @start_time=$$(date +%s%3N); \ + @. /opt/devrail/lib/plugin-execute.sh; \ + start_time=$$(date +%s%3N); \ overall_exit=0; \ ran_languages=""; \ failed_languages=""; \ @@ -872,6 +900,13 @@ _fix: _plugins-load exit $$overall_exit; \ fi; \ fi; \ + dispatch_plugin_target format_fix; \ + if [ "$(DEVRAIL_FAIL_FAST)" = "1" ] && [ $$overall_exit -ne 0 ]; then \ + end_time=$$(date +%s%3N); \ + duration=$$((end_time - start_time)); \ + echo "{\"target\":\"fix\",\"status\":\"fail\",\"duration_ms\":$$duration,\"languages\":[$${ran_languages%,}],\"failed\":[$${failed_languages%,}]}"; \ + exit $$overall_exit; \ + fi; \ end_time=$$(date +%s%3N); \ duration=$$((end_time - start_time)); \ if [ $$overall_exit -eq 0 ]; then \ @@ -883,7 +918,8 @@ _fix: _plugins-load # --- _test: language-specific test runners --- _test: _plugins-load - @start_time=$$(date +%s%3N); \ + @. /opt/devrail/lib/plugin-execute.sh; \ + start_time=$$(date +%s%3N); \ overall_exit=0; \ ran_languages=""; \ failed_languages=""; \ @@ -1051,6 +1087,13 @@ _test: _plugins-load exit $$overall_exit; \ fi; \ fi; \ + dispatch_plugin_target test; \ + if [ "$(DEVRAIL_FAIL_FAST)" = "1" ] && [ $$overall_exit -ne 0 ]; then \ + end_time=$$(date +%s%3N); \ + duration=$$((end_time - start_time)); \ + echo "{\"target\":\"test\",\"status\":\"fail\",\"duration_ms\":$$duration,\"languages\":[$${ran_languages%,}],\"failed\":[$${failed_languages%,}],\"skipped\":[$${skipped_languages%,}]}"; \ + exit $$overall_exit; \ + fi; \ end_time=$$(date +%s%3N); \ duration=$$((end_time - start_time)); \ if [ -z "$${ran_languages}" ] && [ -n "$${skipped_languages}" ]; then \ @@ -1064,7 +1107,8 @@ _test: _plugins-load # --- _security: language-specific security scanners --- _security: _plugins-load - @start_time=$$(date +%s%3N); \ + @. /opt/devrail/lib/plugin-execute.sh; \ + start_time=$$(date +%s%3N); \ overall_exit=0; \ ran_languages=""; \ failed_languages=""; \ @@ -1211,6 +1255,13 @@ _security: _plugins-load exit $$overall_exit; \ fi; \ fi; \ + dispatch_plugin_target security; \ + if [ "$(DEVRAIL_FAIL_FAST)" = "1" ] && [ $$overall_exit -ne 0 ]; then \ + end_time=$$(date +%s%3N); \ + duration=$$((end_time - start_time)); \ + echo "{\"target\":\"security\",\"status\":\"fail\",\"duration_ms\":$$duration,\"languages\":[$${ran_languages%,}],\"failed\":[$${failed_languages%,}]}"; \ + exit $$overall_exit; \ + fi; \ end_time=$$(date +%s%3N); \ duration=$$((end_time - start_time)); \ if [ -z "$${ran_languages}" ] && [ -n "$${skipped_languages}" ]; then \ diff --git a/STABILITY.md b/STABILITY.md index 117e9ae..51e9c80 100644 --- a/STABILITY.md +++ b/STABILITY.md @@ -31,7 +31,7 @@ DevRail has reached **v1.0** across all repositories. The core standards, toolch | **CI workflow templates** | Stable | GitHub Actions workflows and GitLab CI pipeline shipped in template repos. | | **Pre-commit hooks** | Stable | Conventional commit hook and per-language hooks configured in template repos. | | **Documentation site** | Stable | [devrail.dev](https://devrail.dev) is live with full standards coverage. | -| **Plugin loader + resolver + lockfile + build pipeline** | Preview (v1.10.x) | Validates `plugin.devrail.yml` manifests, resolves `rev:` to immutable SHAs (`make plugins-update`), records reproducibility metadata in `.devrail.lock`, and auto-builds a project-local extended image (`devrail-local:`) when plugins are declared. Verifies lockfile + content_hash on every `make check`. No-op when `plugins:` is absent — v1.9.x behaviour unchanged. Plugin command execution (running plugin-defined `targets:`) ships in Story 13.5. | +| **Plugin loader + resolver + lockfile + build pipeline + execution loop** | Preview (v1.10.x) | Validates `plugin.devrail.yml` manifests, resolves `rev:` to immutable SHAs (`make plugins-update`), records reproducibility metadata in `.devrail.lock`, and auto-builds a project-local extended image (`devrail-local:`) when plugins are declared. Each loaded plugin's `targets` are dispatched inside `_lint`/`_format`/`_fix`/`_test`/`_security` with gate evaluation, `{paths}` interpolation, per-language overrides, and JSON aggregation into the existing event shape. `DEVRAIL_FAIL_FAST=1` short-circuits on plugin failures the same as core. No-op when `plugins:` is absent — v1.9.x behaviour unchanged. | ## Consumer responsibilities diff --git a/lib/plugin-execute.sh b/lib/plugin-execute.sh new file mode 100644 index 0000000..9a5cb02 --- /dev/null +++ b/lib/plugin-execute.sh @@ -0,0 +1,256 @@ +#!/usr/bin/env bash +# lib/plugin-execute.sh — Plugin execution dispatcher (Story 13.5) +# +# Purpose: Sourceable library that dispatches each loaded plugin's target +# inside the existing _lint/_format/_fix/_test/_security Makefile +# recipes. Reads from the loader cache (Story 13.2) and updates +# caller-scope shell variables (overall_exit, ran_languages, +# failed_languages) so plugin results aggregate into the same JSON +# shape as core languages. +# +# Usage: source /opt/devrail/lib/plugin-execute.sh +# dispatch_plugin_target lint # or format_check|format_fix|test|security +# +# Contract with Makefile recipes: +# - Caller MUST have these shell vars in scope: overall_exit, ran_languages, +# failed_languages (the recipe's own accounting variables). +# - Caller is responsible for the per-block DEVRAIL_FAIL_FAST short-circuit; +# dispatch_plugin_target itself stops iterating on first failure when +# DEVRAIL_FAIL_FAST=1, and sets overall_exit=1. +# - No-op when the loader cache contains no plugins. +# +# Dependencies: lib/log.sh (log_event), yq (v4+), bash 5+ + +# Guard against double-sourcing +# shellcheck disable=SC2317 +if [[ -n "${_DEVRAIL_PLUGIN_EXECUTE_LOADED:-}" ]]; then + return 0 2>/dev/null || true +fi +readonly _DEVRAIL_PLUGIN_EXECUTE_LOADED=1 + +# Resolve log_event from lib/log.sh if not already sourced. Inside the +# container the canonical path is /opt/devrail/lib; tests can override via +# DEVRAIL_LIB. +if ! declare -f log_event >/dev/null 2>&1; then + # shellcheck source=./log.sh + source "${DEVRAIL_LIB:-/opt/devrail/lib}/log.sh" +fi + +# _devrail_plugin_cache_path returns the cache file path. +# Single helper so future relocation only changes this function. +_devrail_plugin_cache_path() { + printf '%s' "${DEVRAIL_PLUGINS_CACHE:-/tmp/devrail-plugins-loaded.yaml}" +} + +# _devrail_plugin_count returns the number of loaded plugins (0 when cache +# is missing/empty/malformed). Always exits 0 — the dispatcher uses the +# count as a gate, not as a parse-validity signal (the loader has already +# validated the cache by the time the dispatcher runs). +_devrail_plugin_count() { + local cache + cache="$(_devrail_plugin_cache_path)" + if [[ ! -s "${cache}" ]]; then + printf '0' + return 0 + fi + yq -r '.plugins // [] | length' "${cache}" 2>/dev/null || printf '0' +} + +# evaluate_gate +# Returns 0 if every gate path exists (file/dir/glob match), 1 otherwise. +# A skip emits a structured "plugin gate skipped" event listing the missing +# path(s). Empty list or missing key = pass (always run). +# +# Gate path semantics (per design doc § "Manifest schema rules"): +# - Workspace-relative only; absolute paths rejected (config error) +# - Each path matches if it exists as a file OR directory OR has at least +# one glob match (compgen -G) +# - ALL paths must match for the gate to pass +evaluate_gate() { + local idx="${1:?evaluate_gate requires a plugin index}" + local target="${2:?evaluate_gate requires a target name}" + local cache plugin_name gate_count i path missing="" + + cache="$(_devrail_plugin_cache_path)" + plugin_name="$(yq -r ".plugins[${idx}].name // \"\"" "${cache}" 2>/dev/null)" + + gate_count="$(yq -r ".plugins[${idx}].gates.${target} // [] | length" "${cache}" 2>/dev/null || printf '0')" + if [[ "${gate_count}" == "0" ]]; then + return 0 + fi + + for ((i = 0; i < gate_count; i++)); do + path="$(yq -r ".plugins[${idx}].gates.${target}[${i}]" "${cache}" 2>/dev/null)" + if [[ -z "${path}" || "${path}" == "null" ]]; then + continue + fi + if [[ "${path}" == /* ]]; then + log_event error "plugin gate path must be workspace-relative" \ + plugin="${plugin_name}" target="${target}" path="${path}" \ + language=_plugins + return 1 + fi + # File or directory check first (cheap), then glob fallback. + if [[ -e "${path}" ]]; then + continue + fi + if compgen -G "${path}" >/dev/null 2>&1; then + continue + fi + missing="${missing:+${missing}, }${path}" + done + + if [[ -n "${missing}" ]]; then + log_event info "plugin gate skipped" \ + plugin="${plugin_name}" target="${target}" missing="${missing}" \ + language=_plugins + return 1 + fi + return 0 +} + +# render_cmd +# Prints the rendered command string on stdout. +# Substitutes {paths} with the runtime value of ${} filtered to +# existing paths (mirroring how RUBY_PATHS is filtered in the Ruby block of +# the core Makefile). Falls back to paths_default if the env var is unset. +# When cmd contains literal `{paths}` but no paths_var is declared, exits 2 +# with a structured error (configuration mistake). +render_cmd() { + local idx="${1:?render_cmd requires a plugin index}" + local target="${2:?render_cmd requires a target name}" + local cache cmd paths_var paths_default plugin_name p filtered="" + + cache="$(_devrail_plugin_cache_path)" + plugin_name="$(yq -r ".plugins[${idx}].name // \"\"" "${cache}" 2>/dev/null)" + cmd="$(yq -r ".plugins[${idx}].targets.${target}.cmd // \"\"" "${cache}" 2>/dev/null)" + + if [[ "${cmd}" != *"{paths}"* ]]; then + printf '%s' "${cmd}" + return 0 + fi + + paths_var="$(yq -r ".plugins[${idx}].targets.${target}.paths_var // \"\"" "${cache}" 2>/dev/null)" + paths_default="$(yq -r ".plugins[${idx}].targets.${target}.paths_default // \"\"" "${cache}" 2>/dev/null)" + + if [[ -z "${paths_var}" || "${paths_var}" == "null" ]]; then + log_event error "plugin cmd uses {paths} but declares no paths_var" \ + plugin="${plugin_name}" target="${target}" cmd="${cmd}" \ + language=_plugins + exit 2 + fi + + local raw="${!paths_var:-${paths_default}}" + for p in ${raw}; do + if [[ -e "${p}" ]]; then + filtered="${filtered:+${filtered} }${p}" + fi + done + + printf '%s' "${cmd//\{paths\}/${filtered}}" +} + +# apply_override +# Prints the user-supplied override from .devrail.yml when present, otherwise +# echoes the default cmd back. Override key by target: +# lint → linter +# format_check → formatter +# format_fix → formatter (same as format_check; symmetric with core) +# fix → fixer +# test → test +# security → security +# The override replaces the entire cmd string (no {paths} interpolation). +apply_override() { + local language="${1:?apply_override requires a language}" + local target="${2:?apply_override requires a target name}" + local default_cmd="${3:-}" + local devrail_yml="${DEVRAIL_CONFIG:-/workspace/.devrail.yml}" + local key override + + if [[ ! -r "${devrail_yml}" ]]; then + printf '%s' "${default_cmd}" + return 0 + fi + + case "${target}" in + lint) key="linter" ;; + format_check | format_fix) key="formatter" ;; + fix) key="fixer" ;; + test) key="test" ;; + security) key="security" ;; + *) + printf '%s' "${default_cmd}" + return 0 + ;; + esac + + override="$(LANG_KEY="${language}" KEY="${key}" yq -r '.[strenv(LANG_KEY)][strenv(KEY)] // ""' "${devrail_yml}" 2>/dev/null)" + if [[ -n "${override}" && "${override}" != "null" ]]; then + printf '%s' "${override}" + return 0 + fi + printf '%s' "${default_cmd}" +} + +# dispatch_plugin_target +# Iterates over loaded plugins; for each plugin that defines this target, +# evaluates the gate, renders the command, applies any per-language override, +# and runs the cmd via `bash -c`. Updates caller-scope shell variables +# (overall_exit, ran_languages, failed_languages) and honours +# DEVRAIL_FAIL_FAST=1 by returning early after the first failure. +# +# No-op when no plugins are loaded. +dispatch_plugin_target() { + local target="${1:?dispatch_plugin_target requires a target name}" + local plugin_count i name cmd rendered final_cmd plugin_exit + + plugin_count="$(_devrail_plugin_count)" + if [[ "${plugin_count}" == "0" ]]; then + return 0 + fi + + local cache + cache="$(_devrail_plugin_cache_path)" + + for ((i = 0; i < plugin_count; i++)); do + name="$(yq -r ".plugins[${i}].name" "${cache}")" + cmd="$(yq -r ".plugins[${i}].targets.${target}.cmd // \"\"" "${cache}" 2>/dev/null)" + if [[ -z "${cmd}" || "${cmd}" == "null" ]]; then + # Plugin doesn't expose this target — silent skip (no event). + continue + fi + + if ! evaluate_gate "${i}" "${target}"; then + continue + fi + + rendered="$(render_cmd "${i}" "${target}")" + final_cmd="$(apply_override "${name}" "${target}" "${rendered}")" + + log_event info "plugin target executing" \ + plugin="${name}" target="${target}" language=_plugins + plugin_exit=0 + bash -c "${final_cmd}" || plugin_exit=$? + if [[ "${plugin_exit}" -eq 0 ]]; then + # shellcheck disable=SC2034 # caller-scope variable from Makefile recipe + ran_languages="${ran_languages}\"${name}\"," + log_event info "plugin target passed" \ + plugin="${name}" target="${target}" language=_plugins + else + # shellcheck disable=SC2034 # caller-scope variables from Makefile recipe + ran_languages="${ran_languages}\"${name}\"," + # shellcheck disable=SC2034 + failed_languages="${failed_languages}\"${name}\"," + # shellcheck disable=SC2034 + overall_exit=1 + log_event error "plugin target failed" \ + plugin="${name}" target="${target}" exit_code:="${plugin_exit}" \ + language=_plugins + if [[ "${DEVRAIL_FAIL_FAST}" == "1" ]]; then + return 1 + fi + fi + done + + return 0 +} diff --git a/tests/test-plugin-execution.sh b/tests/test-plugin-execution.sh new file mode 100755 index 0000000..e39153a --- /dev/null +++ b/tests/test-plugin-execution.sh @@ -0,0 +1,375 @@ +#!/usr/bin/env bash +# tests/test-plugin-execution.sh — Validate the plugin execution loop (Story 13.5) +# +# 10 cases covering AC10: +# 1. No plugins → dispatcher is a no-op (no events, no aggregation) +# 2. Single plugin, lint passes → entry in ran_languages, exit 0 +# 3. Single plugin, lint fails → entry in failed_languages, overall_exit=1 +# 4. Gate skip (gate path absent) → no execution, 'gate skipped' event +# 5. Gate run (gate path present) → executes normally +# 6. {paths} interpolation with paths_var (filtered to existing paths) +# 7. Per-language override replaces manifest default cmd +# 8. DEVRAIL_FAIL_FAST=1 short-circuits on first plugin failure +# 9. Manifest has 'lint' but no 'test' target → _test invocation skips silently +# 10. JSON regression: zero-plugin run produces byte-identical event shape +# +# Usage: bash tests/test-plugin-execution.sh +# Env: +# DEVRAIL_IMAGE override image name (default: ghcr.io/devrail-dev/dev-toolchain) +# DEVRAIL_TAG override image tag (default: local) + +set -euo pipefail + +IMAGE="${DEVRAIL_IMAGE:-ghcr.io/devrail-dev/dev-toolchain}:${DEVRAIL_TAG:-local}" +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +WORKDIR="$(mktemp -d)" + +cleanup() { + if [ -n "${WORKDIR:-}" ] && [ -d "$WORKDIR" ]; then + docker run --rm -v "$WORKDIR:/cleanup" "$IMAGE" \ + sh -c 'rm -rf /cleanup/* /cleanup/.[!.]* 2>/dev/null || true' >/dev/null 2>&1 || true + rmdir "$WORKDIR" 2>/dev/null || rm -rf "$WORKDIR" 2>/dev/null || true + fi +} +trap cleanup EXIT + +assert_eq() { + local expected="$1" actual="$2" context="$3" + if [ "$expected" != "$actual" ]; then + echo "FAIL [$context]: expected '$expected', got '$actual'" >&2 + exit 1 + fi +} + +assert_jq() { + local file="$1" filter="$2" context="$3" + if ! grep -aE '^\{.*\}$' "$file" | jq -e "$filter" >/dev/null 2>&1; then + echo "FAIL [$context]: jq filter '$filter' did not match any event in:" >&2 + cat "$file" >&2 + exit 1 + fi +} + +# run_dispatch_in_container +# Sources lib/plugin-execute.sh inside the container and calls +# dispatch_plugin_target. Stdout = the final shell vars; stderr = log_event +# stream. Exit code = overall_exit at end of dispatch. +run_dispatch_in_container() { + local case_dir="$1" target="$2" + RUN_EXIT=0 + RUN_OUT="$(docker run --rm \ + -v "$case_dir:/workspace" \ + -v "$REPO_ROOT/lib:/opt/devrail/lib:ro" \ + -e DEVRAIL_PLUGINS_CACHE=/workspace/cache.yaml \ + -e DEVRAIL_LOG_FORMAT=json \ + -e DEVRAIL_FAIL_FAST="${DEVRAIL_FAIL_FAST:-0}" \ + -e DEVRAIL_CONFIG=/workspace/.devrail.yml \ + -w /workspace \ + "$IMAGE" \ + bash -c ' + set +e + . /opt/devrail/lib/plugin-execute.sh + overall_exit=0 + ran_languages="" + failed_languages="" + dispatch_plugin_target "'"$target"'" + echo "DISPATCH_RESULT overall_exit=$overall_exit ran_languages=${ran_languages%,} failed_languages=${failed_languages%,}" + exit $overall_exit + ' 2>&1)" || RUN_EXIT=$? +} + +# --- Case 1: no plugins → dispatcher is a no-op --- +echo "==> Case 1: empty cache → dispatcher no-op, exit 0" +mkdir -p "$WORKDIR/case1" +cat >"$WORKDIR/case1/cache.yaml" <<'YAML' +plugins: [] +YAML +cat >"$WORKDIR/case1/.devrail.yml" <<'YAML' +languages: [bash] +YAML +run_dispatch_in_container "$WORKDIR/case1" "lint" +assert_eq "0" "$RUN_EXIT" "case1 exit code" +echo "$RUN_OUT" | grep -q "DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages=" || { + echo "FAIL [case1]: expected empty ran/failed_languages, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +# No plugin-related events should appear +if echo "$RUN_OUT" | grep -q "plugin target"; then + echo "FAIL [case1]: dispatcher should not emit plugin events when cache is empty" >&2 + echo "$RUN_OUT" >&2 + exit 1 +fi + +# --- Case 2: single plugin, lint passes → ran_languages updated --- +echo "==> Case 2: single plugin, lint cmd 'true' → ran_languages=elixir" +mkdir -p "$WORKDIR/case2" +cat >"$WORKDIR/case2/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "true" +YAML +cat >"$WORKDIR/case2/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +run_dispatch_in_container "$WORKDIR/case2" "lint" +assert_eq "0" "$RUN_EXIT" "case2 exit code" +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages="elixir" failed_languages=$' || { + echo "FAIL [case2]: expected ran_languages=\"elixir\", got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +echo "$RUN_OUT" >"$WORKDIR/case2.log" +assert_jq "$WORKDIR/case2.log" 'select(.msg=="plugin target executing" and .plugin=="elixir" and .target=="lint")' "case2 executing event" +assert_jq "$WORKDIR/case2.log" 'select(.msg=="plugin target passed" and .plugin=="elixir")' "case2 passed event" + +# --- Case 3: single plugin, lint fails → failed_languages updated --- +echo "==> Case 3: single plugin, lint cmd 'false' → failed_languages=elixir" +mkdir -p "$WORKDIR/case3" +cat >"$WORKDIR/case3/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "false" +YAML +cat >"$WORKDIR/case3/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +run_dispatch_in_container "$WORKDIR/case3" "lint" +assert_eq "1" "$RUN_EXIT" "case3 exit code" +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=1 ran_languages="elixir" failed_languages="elixir"$' || { + echo "FAIL [case3]: expected failed_languages=\"elixir\", got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +echo "$RUN_OUT" >"$WORKDIR/case3.log" +assert_jq "$WORKDIR/case3.log" 'select(.level=="error" and .msg=="plugin target failed" and .plugin=="elixir")' "case3 failed event" + +# --- Case 4: gate path absent → skip without execution --- +echo "==> Case 4: gate path absent → skip event, no execution" +mkdir -p "$WORKDIR/case4" +cat >"$WORKDIR/case4/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "false" + gates: + lint: ["mix.exs"] +YAML +cat >"$WORKDIR/case4/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +run_dispatch_in_container "$WORKDIR/case4" "lint" +assert_eq "0" "$RUN_EXIT" "case4 exit code" +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages=$' || { + echo "FAIL [case4]: gate skip should leave ran/failed empty, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +echo "$RUN_OUT" >"$WORKDIR/case4.log" +assert_jq "$WORKDIR/case4.log" 'select(.msg=="plugin gate skipped" and .plugin=="elixir" and .target=="lint")' "case4 gate skipped event" + +# --- Case 5: gate path present → executes normally --- +echo "==> Case 5: gate path present → executes" +mkdir -p "$WORKDIR/case5" +touch "$WORKDIR/case5/mix.exs" +cat >"$WORKDIR/case5/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "true" + gates: + lint: ["mix.exs"] +YAML +cat >"$WORKDIR/case5/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +run_dispatch_in_container "$WORKDIR/case5" "lint" +assert_eq "0" "$RUN_EXIT" "case5 exit code" +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages="elixir" failed_languages=$' || { + echo "FAIL [case5]: gate-passed lint should record ran_languages, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} + +# --- Case 6: {paths} interpolation --- +echo "==> Case 6: {paths} interpolated from paths_var (existing paths only)" +mkdir -p "$WORKDIR/case6/lib" "$WORKDIR/case6/test" +# Note: we set ELIXIR_PATHS to include a non-existent path; the dispatcher +# must filter it out before interpolation. +cat >"$WORKDIR/case6/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "echo PATHS={paths} > /workspace/result.txt" + paths_var: ELIXIR_PATHS + paths_default: "lib test" +YAML +cat >"$WORKDIR/case6/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +RUN_EXIT=0 +RUN_OUT="$(docker run --rm \ + -v "$WORKDIR/case6:/workspace" \ + -v "$REPO_ROOT/lib:/opt/devrail/lib:ro" \ + -e DEVRAIL_PLUGINS_CACHE=/workspace/cache.yaml \ + -e DEVRAIL_LOG_FORMAT=json \ + -e DEVRAIL_CONFIG=/workspace/.devrail.yml \ + -e ELIXIR_PATHS="lib test does-not-exist" \ + -w /workspace \ + "$IMAGE" \ + bash -c ' + set +e + . /opt/devrail/lib/plugin-execute.sh + overall_exit=0; ran_languages=""; failed_languages="" + dispatch_plugin_target lint + exit $overall_exit + ' 2>&1)" || RUN_EXIT=$? +assert_eq "0" "$RUN_EXIT" "case6 exit code" +result="$(cat "$WORKDIR/case6/result.txt" 2>/dev/null || true)" +assert_eq "PATHS=lib test" "$result" "case6 paths interpolated and filtered" + +# --- Case 7: per-language override replaces manifest default --- +echo "==> Case 7: .devrail.yml override replaces plugin default cmd" +mkdir -p "$WORKDIR/case7" +cat >"$WORKDIR/case7/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "false" +YAML +# Override: replace 'false' with 'true' so target passes. +cat >"$WORKDIR/case7/.devrail.yml" <<'YAML' +languages: [elixir] +elixir: + linter: "true" +YAML +run_dispatch_in_container "$WORKDIR/case7" "lint" +assert_eq "0" "$RUN_EXIT" "case7 exit code (override 'false' → 'true')" +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages="elixir"' || { + echo "FAIL [case7]: override should make plugin target pass, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} + +# --- Case 8: DEVRAIL_FAIL_FAST=1 short-circuits --- +echo "==> Case 8: DEVRAIL_FAIL_FAST=1 stops on first plugin failure" +mkdir -p "$WORKDIR/case8" +# Two plugins; first fails, second has cmd 'touch /workspace/second-ran' so +# the test can detect whether it was reached. +cat >"$WORKDIR/case8/cache.yaml" <<'YAML' +plugins: + - name: alpha + rev: v1.0.0 + source: github.com/test/alpha + schema_version: 1 + targets: + lint: + cmd: "false" + - name: beta + rev: v1.0.0 + source: github.com/test/beta + schema_version: 1 + targets: + lint: + cmd: "touch /workspace/second-ran" +YAML +cat >"$WORKDIR/case8/.devrail.yml" <<'YAML' +languages: [alpha, beta] +YAML +DEVRAIL_FAIL_FAST=1 run_dispatch_in_container "$WORKDIR/case8" "lint" +assert_eq "1" "$RUN_EXIT" "case8 exit code" +[ ! -f "$WORKDIR/case8/second-ran" ] || { + echo "FAIL [case8]: second plugin must NOT run under DEVRAIL_FAIL_FAST=1" >&2 + exit 1 +} + +# --- Case 9: manifest has 'lint' but not 'test' --- +echo "==> Case 9: plugin without 'test' target → _test dispatch silent skip" +mkdir -p "$WORKDIR/case9" +cat >"$WORKDIR/case9/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "true" +YAML +cat >"$WORKDIR/case9/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +run_dispatch_in_container "$WORKDIR/case9" "test" +assert_eq "0" "$RUN_EXIT" "case9 exit code (no 'test' target)" +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages=$' || { + echo "FAIL [case9]: silent skip should leave ran/failed empty, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +# No plugin events for the test target +if echo "$RUN_OUT" | grep -q "plugin target executing"; then + echo "FAIL [case9]: should not emit 'executing' event when target absent" >&2 + exit 1 +fi + +# --- Case 10: zero-plugin _lint emits a baseline JSON shape --- +echo "==> Case 10: empty cache + _lint envelope → identical JSON to v1.10.4 baseline" +mkdir -p "$WORKDIR/case10" +cat >"$WORKDIR/case10/cache.yaml" <<'YAML' +plugins: [] +YAML +cat >"$WORKDIR/case10/.devrail.yml" <<'YAML' +languages: [bash] +YAML +RUN_EXIT=0 +case10_out="$(docker run --rm \ + -v "$WORKDIR/case10:/workspace" \ + -v "$REPO_ROOT/lib:/opt/devrail/lib:ro" \ + -e DEVRAIL_PLUGINS_CACHE=/workspace/cache.yaml \ + -e DEVRAIL_LOG_FORMAT=json \ + -e DEVRAIL_CONFIG=/workspace/.devrail.yml \ + -w /workspace \ + "$IMAGE" \ + bash -c ' + set +e + . /opt/devrail/lib/plugin-execute.sh + overall_exit=0; ran_languages=""; failed_languages="" + dispatch_plugin_target lint + # Emit the same JSON shape the recipe would emit (pass path) + echo "{\"target\":\"lint\",\"status\":\"pass\",\"duration_ms\":0,\"languages\":[${ran_languages%,}]}" + exit $overall_exit + ' 2>&1)" || RUN_EXIT=$? +assert_eq "0" "$RUN_EXIT" "case10 exit code" +# The JSON line must match the v1.10.x shape exactly: no `plugins:` array, no +# extra fields. `languages: []` is the baseline shape for "no languages". +expected_shape='{"target":"lint","status":"pass","duration_ms":0,"languages":[]}' +got_line="$(echo "$case10_out" | grep -E '^\{.*"target":"lint"' | tail -1)" +assert_eq "$expected_shape" "$got_line" "case10 JSON shape regression" + +echo "==> All plugin-execution checks passed (10/10)" From f246ab47decb6e1d13f80e275bc08c0b11f3bfe2 Mon Sep 17 00:00:00 2001 From: Matthew Mellor Date: Mon, 4 May 2026 21:12:56 -0500 Subject: [PATCH 2/2] fix(makefile): address Story 13.5 senior-developer review findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit H1: render_cmd no longer calls `exit 2` from the sourced library — that killed the recipe before the JSON envelope emitted. Helpers now return non-zero, dispatcher catches them, marks the plugin failed, and lets the recipe continue (Case 12 verifies the second plugin still runs). M1: evaluate_gate now returns 0/1/2 distinctly. Gate-skip (path missing) is silent; absolute-path config error surfaces as plugin failure with a `:gate-config` entry in failed_languages (Case 11). M2: yq parse errors on the loader cache are no longer swallowed by `2>/dev/null`. A malformed cache produces a structured error event and a `_plugins:cache-parse` plugin-system failure (Case 15). M3: cache + .devrail.yml are converted to JSON once at the start of each dispatch and re-used via jq for all per-plugin lookups. Earlier path invoked yq ~8N times per recipe; new path is 2 yq calls + ~3N jq calls (each jq invocation on the cached JSON is much faster than yq on the YAML file). M4: optional DEVRAIL_PLUGIN_TIMEOUT_SECONDS env wraps each plugin cmd in `timeout -k 5 N bash -c …`. Unset = no timeout (default). M5: _fix dispatches both `format_fix` AND `fix` so plugins that declare `targets.fix.cmd:` per the schema actually run. M6: render_cmd's filter loop rejects path entries containing shell-meta characters (`;|&$<>(){}\` etc.) with a structured warn event. Closes the path-with-`;` injection vector through `bash -c "${final_cmd}"` (Case 13 inserts a `lib;evil` directory and asserts it's filtered). L1: caller-scope vars registered with shellcheck via a single `:` no-op assignment at the top of dispatch_plugin_target. Drops 4 scattered SC2034 disables. L2: 5 new test cases (11-15) cover absolute-path gate, {paths} without paths_var, shell-meta path rejection, triple-source idempotency, and malformed-cache parse error. L3: Case 9 (plugin without 'test' target) tightens silent-skip assertion — now rejects ANY plugin event for the absent target. L4: STABILITY.md notes the Makefile contract pins SHELL := /bin/bash as of v1.10.x for consumer template repos that inherit it. L5: dispatcher appends gate-skipped plugins to skipped_languages so _test/_security recipes that maintain that array stay consistent with how core "no work to do" cases are reported. Co-Authored-By: Claude Opus 4.7 (1M context) --- Makefile | 7 + STABILITY.md | 2 +- lib/plugin-execute.sh | 262 +++++++++++++++++++++------------ tests/test-plugin-execution.sh | 226 +++++++++++++++++++++++++--- 4 files changed, 384 insertions(+), 113 deletions(-) diff --git a/Makefile b/Makefile index e21084d..ab6e288 100644 --- a/Makefile +++ b/Makefile @@ -907,6 +907,13 @@ _fix: _plugins-load echo "{\"target\":\"fix\",\"status\":\"fail\",\"duration_ms\":$$duration,\"languages\":[$${ran_languages%,}],\"failed\":[$${failed_languages%,}]}"; \ exit $$overall_exit; \ fi; \ + dispatch_plugin_target fix; \ + if [ "$(DEVRAIL_FAIL_FAST)" = "1" ] && [ $$overall_exit -ne 0 ]; then \ + end_time=$$(date +%s%3N); \ + duration=$$((end_time - start_time)); \ + echo "{\"target\":\"fix\",\"status\":\"fail\",\"duration_ms\":$$duration,\"languages\":[$${ran_languages%,}],\"failed\":[$${failed_languages%,}]}"; \ + exit $$overall_exit; \ + fi; \ end_time=$$(date +%s%3N); \ duration=$$((end_time - start_time)); \ if [ $$overall_exit -eq 0 ]; then \ diff --git a/STABILITY.md b/STABILITY.md index 51e9c80..f9bdab2 100644 --- a/STABILITY.md +++ b/STABILITY.md @@ -14,7 +14,7 @@ DevRail has reached **v1.0** across all repositories. The core standards, toolch | Component | Status | Notes | |---|---|---| | **Container image** | Stable | Multi-arch (amd64 + arm64), signed with cosign, weekly rebuilds. | -| **Makefile contract** | Stable | Two-layer delegation pattern, JSON summary output, `init` scaffolding. | +| **Makefile contract** | Stable | Two-layer delegation pattern, JSON summary output, `init` scaffolding. As of v1.10.x the reference Makefile pins `SHELL := /bin/bash` so plugin libraries can be sourced directly into recipes; consumer template repos that inherit this Makefile require `bash` on the host (already the default on Debian/Ubuntu/macOS — only relevant for busybox/Alpine without bash). | | **Shell conventions** | Stable | `lib/log.sh`, `lib/platform.sh`, header format, and idempotency patterns are settled. | | **Conventional commits** | Stable | Types, scopes, and format are finalized. Pre-commit hook published. | | **Language standards** | Stable | Python, Bash, Terraform, Ansible, Ruby, Go, JavaScript/TypeScript, Rust — all 8 ecosystems shipped. | diff --git a/lib/plugin-execute.sh b/lib/plugin-execute.sh index 9a5cb02..776d888 100644 --- a/lib/plugin-execute.sh +++ b/lib/plugin-execute.sh @@ -5,23 +5,37 @@ # inside the existing _lint/_format/_fix/_test/_security Makefile # recipes. Reads from the loader cache (Story 13.2) and updates # caller-scope shell variables (overall_exit, ran_languages, -# failed_languages) so plugin results aggregate into the same JSON -# shape as core languages. +# failed_languages, skipped_languages) so plugin results aggregate +# into the same JSON shape as core languages. # # Usage: source /opt/devrail/lib/plugin-execute.sh -# dispatch_plugin_target lint # or format_check|format_fix|test|security +# dispatch_plugin_target lint # or format_check|format_fix|fix|test|security # # Contract with Makefile recipes: # - Caller MUST have these shell vars in scope: overall_exit, ran_languages, -# failed_languages (the recipe's own accounting variables). +# failed_languages (the recipe's own accounting variables). For _test and +# _security recipes, skipped_languages is also updated on gate-skip. # - Caller is responsible for the per-block DEVRAIL_FAIL_FAST short-circuit; # dispatch_plugin_target itself stops iterating on first failure when # DEVRAIL_FAIL_FAST=1, and sets overall_exit=1. # - No-op when the loader cache contains no plugins. +# - Internal helpers MUST NOT exit; they return non-zero so the dispatcher +# can surface failures via the recipe's JSON envelope (review H1). # -# Dependencies: lib/log.sh (log_event), yq (v4+), bash 5+ +# Performance: +# The cache and `.devrail.yml` are converted to JSON once at the start of +# each dispatch and re-used via jq for all per-plugin / per-target lookups. +# Earlier revisions invoked yq ~8N times per recipe (review M3); the +# current path is 2 yq calls + ~3N jq calls. +# +# Optional env: +# DEVRAIL_PLUGIN_TIMEOUT_SECONDS — wraps each plugin cmd in `timeout -k 5 N` +# (review M4). Unset = no timeout (default). +# +# Dependencies: lib/log.sh (log_event), yq (v4+), jq, bash 5+, coreutils +# (timeout — only when DEVRAIL_PLUGIN_TIMEOUT_SECONDS is set) -# Guard against double-sourcing +# Guard against double-sourcing (review L2 — covered by tests case 14) # shellcheck disable=SC2317 if [[ -n "${_DEVRAIL_PLUGIN_EXECUTE_LOADED:-}" ]]; then return 0 2>/dev/null || true @@ -37,58 +51,45 @@ if ! declare -f log_event >/dev/null 2>&1; then fi # _devrail_plugin_cache_path returns the cache file path. -# Single helper so future relocation only changes this function. _devrail_plugin_cache_path() { printf '%s' "${DEVRAIL_PLUGINS_CACHE:-/tmp/devrail-plugins-loaded.yaml}" } -# _devrail_plugin_count returns the number of loaded plugins (0 when cache -# is missing/empty/malformed). Always exits 0 — the dispatcher uses the -# count as a gate, not as a parse-validity signal (the loader has already -# validated the cache by the time the dispatcher runs). -_devrail_plugin_count() { - local cache - cache="$(_devrail_plugin_cache_path)" - if [[ ! -s "${cache}" ]]; then - printf '0' - return 0 - fi - yq -r '.plugins // [] | length' "${cache}" 2>/dev/null || printf '0' -} +# Note: the cache + .devrail.yml load is inlined into dispatch_plugin_target +# rather than factored into a helper. A helper that returned both JSON blobs +# via `printf -v ` would shadow the caller's var when both used the +# same name (bash's `local` scoping breaks indirect assignment that way), so +# we keep the parse inline. The single yq → JSON conversion still happens +# only once per dispatch (review M3). -# evaluate_gate -# Returns 0 if every gate path exists (file/dir/glob match), 1 otherwise. -# A skip emits a structured "plugin gate skipped" event listing the missing -# path(s). Empty list or missing key = pass (always run). +# evaluate_gate +# Returns: +# 0 — gate passed (target should run) +# 1 — gate skip (path missing); emits structured info event +# 2 — gate config error (absolute path); emits structured error event # -# Gate path semantics (per design doc § "Manifest schema rules"): -# - Workspace-relative only; absolute paths rejected (config error) -# - Each path matches if it exists as a file OR directory OR has at least -# one glob match (compgen -G) -# - ALL paths must match for the gate to pass +# The caller treats 1 as silent skip, 2 as plugin failure (review M1). evaluate_gate() { - local idx="${1:?evaluate_gate requires a plugin index}" - local target="${2:?evaluate_gate requires a target name}" - local cache plugin_name gate_count i path missing="" - - cache="$(_devrail_plugin_cache_path)" - plugin_name="$(yq -r ".plugins[${idx}].name // \"\"" "${cache}" 2>/dev/null)" + local cache_json="${1:?evaluate_gate requires cache JSON}" + local idx="${2:?evaluate_gate requires a plugin index}" + local target="${3:?evaluate_gate requires a target name}" + local plugin_name="${4:?evaluate_gate requires a plugin name}" + local gates path missing="" - gate_count="$(yq -r ".plugins[${idx}].gates.${target} // [] | length" "${cache}" 2>/dev/null || printf '0')" - if [[ "${gate_count}" == "0" ]]; then + gates="$(IDX="${idx}" TARGET="${target}" \ + jq -r '.plugins[env.IDX|tonumber].gates[env.TARGET] // [] | .[]' \ + <<<"${cache_json}")" + if [[ -z "${gates}" ]]; then return 0 fi - for ((i = 0; i < gate_count; i++)); do - path="$(yq -r ".plugins[${idx}].gates.${target}[${i}]" "${cache}" 2>/dev/null)" - if [[ -z "${path}" || "${path}" == "null" ]]; then - continue - fi + while IFS= read -r path; do + [[ -z "${path}" || "${path}" == "null" ]] && continue if [[ "${path}" == /* ]]; then log_event error "plugin gate path must be workspace-relative" \ plugin="${plugin_name}" target="${target}" path="${path}" \ language=_plugins - return 1 + return 2 fi # File or directory check first (cheap), then glob fallback. if [[ -e "${path}" ]]; then @@ -98,7 +99,7 @@ evaluate_gate() { continue fi missing="${missing:+${missing}, }${path}" - done + done <<<"${gates}" if [[ -n "${missing}" ]]; then log_event info "plugin gate skipped" \ @@ -109,39 +110,52 @@ evaluate_gate() { return 0 } -# render_cmd -# Prints the rendered command string on stdout. -# Substitutes {paths} with the runtime value of ${} filtered to -# existing paths (mirroring how RUBY_PATHS is filtered in the Ruby block of -# the core Makefile). Falls back to paths_default if the env var is unset. -# When cmd contains literal `{paths}` but no paths_var is declared, exits 2 -# with a structured error (configuration mistake). +# render_cmd +# Prints the rendered command on stdout. `{paths}` is substituted with the +# value of `${}` filtered to existing paths whose names contain +# no shell-meta characters (review M6). Returns: +# 0 — rendered successfully +# 2 — config error (`{paths}` referenced but `paths_var` not declared); +# emits structured error event (review H1: returns instead of exit) render_cmd() { - local idx="${1:?render_cmd requires a plugin index}" - local target="${2:?render_cmd requires a target name}" - local cache cmd paths_var paths_default plugin_name p filtered="" - - cache="$(_devrail_plugin_cache_path)" - plugin_name="$(yq -r ".plugins[${idx}].name // \"\"" "${cache}" 2>/dev/null)" - cmd="$(yq -r ".plugins[${idx}].targets.${target}.cmd // \"\"" "${cache}" 2>/dev/null)" + local cache_json="${1:?render_cmd requires cache JSON}" + local idx="${2:?render_cmd requires a plugin index}" + local target="${3:?render_cmd requires a target name}" + local plugin_name="${4:?render_cmd requires a plugin name}" + local cmd="${5:-}" + local paths_var paths_default raw p filtered="" if [[ "${cmd}" != *"{paths}"* ]]; then printf '%s' "${cmd}" return 0 fi - paths_var="$(yq -r ".plugins[${idx}].targets.${target}.paths_var // \"\"" "${cache}" 2>/dev/null)" - paths_default="$(yq -r ".plugins[${idx}].targets.${target}.paths_default // \"\"" "${cache}" 2>/dev/null)" + paths_var="$(IDX="${idx}" TARGET="${target}" \ + jq -r '.plugins[env.IDX|tonumber].targets[env.TARGET].paths_var // ""' \ + <<<"${cache_json}")" + paths_default="$(IDX="${idx}" TARGET="${target}" \ + jq -r '.plugins[env.IDX|tonumber].targets[env.TARGET].paths_default // ""' \ + <<<"${cache_json}")" if [[ -z "${paths_var}" || "${paths_var}" == "null" ]]; then log_event error "plugin cmd uses {paths} but declares no paths_var" \ plugin="${plugin_name}" target="${target}" cmd="${cmd}" \ language=_plugins - exit 2 + return 2 fi - local raw="${!paths_var:-${paths_default}}" + raw="${!paths_var:-${paths_default}}" for p in ${raw}; do + # Reject shell-meta chars to keep `bash -c "${final_cmd}"` injection-free + # when paths come from user-controlled env vars (review M6). + case "${p}" in + *[\;\|\&\$\<\>\(\)\`\\\"\']*) + log_event warn "plugin path contains shell-meta characters; skipping" \ + plugin="${plugin_name}" target="${target}" path="${p}" \ + language=_plugins + continue + ;; + esac if [[ -e "${p}" ]]; then filtered="${filtered:+${filtered} }${p}" fi @@ -150,28 +164,26 @@ render_cmd() { printf '%s' "${cmd//\{paths\}/${filtered}}" } -# apply_override -# Prints the user-supplied override from .devrail.yml when present, otherwise -# echoes the default cmd back. Override key by target: +# apply_override +# Prints the user-supplied override from `.devrail.yml` when present, +# otherwise echoes the default cmd back. +# +# Override key by target: # lint → linter # format_check → formatter # format_fix → formatter (same as format_check; symmetric with core) # fix → fixer # test → test # security → security -# The override replaces the entire cmd string (no {paths} interpolation). +# +# The override replaces the entire cmd string (no `{paths}` interpolation). apply_override() { - local language="${1:?apply_override requires a language}" - local target="${2:?apply_override requires a target name}" - local default_cmd="${3:-}" - local devrail_yml="${DEVRAIL_CONFIG:-/workspace/.devrail.yml}" + local devrail_json="${1:?apply_override requires devrail JSON}" + local language="${2:?apply_override requires a language}" + local target="${3:?apply_override requires a target name}" + local default_cmd="${4:-}" local key override - if [[ ! -r "${devrail_yml}" ]]; then - printf '%s' "${default_cmd}" - return 0 - fi - case "${target}" in lint) key="linter" ;; format_check | format_fix) key="formatter" ;; @@ -184,7 +196,8 @@ apply_override() { ;; esac - override="$(LANG_KEY="${language}" KEY="${key}" yq -r '.[strenv(LANG_KEY)][strenv(KEY)] // ""' "${devrail_yml}" 2>/dev/null)" + override="$(LANG_KEY="${language}" KEY="${key}" \ + jq -r '.[env.LANG_KEY][env.KEY] // ""' <<<"${devrail_json}")" if [[ -n "${override}" && "${override}" != "null" ]]; then printf '%s' "${override}" return 0 @@ -195,53 +208,112 @@ apply_override() { # dispatch_plugin_target # Iterates over loaded plugins; for each plugin that defines this target, # evaluates the gate, renders the command, applies any per-language override, -# and runs the cmd via `bash -c`. Updates caller-scope shell variables -# (overall_exit, ran_languages, failed_languages) and honours -# DEVRAIL_FAIL_FAST=1 by returning early after the first failure. +# and runs the cmd via `bash -c` (optionally wrapped in `timeout` when +# DEVRAIL_PLUGIN_TIMEOUT_SECONDS is set). Updates caller-scope shell +# variables and honours DEVRAIL_FAIL_FAST=1. +# +# Caller-scope vars updated: +# overall_exit — set to 1 on plugin failure (or config error) +# ran_languages — appended on success or failure (the plugin ran) +# failed_languages — appended on cmd failure or config error +# skipped_languages — appended on gate-skip (review L5; harmless when +# the recipe doesn't use it) # # No-op when no plugins are loaded. dispatch_plugin_target() { local target="${1:?dispatch_plugin_target requires a target name}" - local plugin_count i name cmd rendered final_cmd plugin_exit - plugin_count="$(_devrail_plugin_count)" - if [[ "${plugin_count}" == "0" ]]; then + # Register caller-scope vars so shellcheck doesn't flag the writes below + # as SC2034 (review L1). The `:` builtin and `:=` default-assign keep the + # vars alive in the caller's shell without overwriting existing values. + : "${overall_exit:=0}" "${ran_languages:=}" "${failed_languages:=}" "${skipped_languages:=}" + + local cache cache_json devrail_yml devrail_json="{}" parsed plugin_count i + cache="$(_devrail_plugin_cache_path)" + if [[ ! -s "${cache}" ]]; then return 0 fi + if ! cache_json="$(yq -o=json . "${cache}" 2>&1)"; then + log_event error "loader cache could not be parsed by yq" \ + cache="${cache}" stderr="${cache_json}" language=_plugins + overall_exit=1 + failed_languages="${failed_languages}\"_plugins:cache-parse\"," + return 1 + fi + devrail_yml="${DEVRAIL_CONFIG:-/workspace/.devrail.yml}" + if [[ -r "${devrail_yml}" ]]; then + if parsed="$(yq -o=json . "${devrail_yml}" 2>&1)"; then + devrail_json="${parsed}" + else + log_event warn ".devrail.yml could not be parsed; per-language overrides disabled" \ + path="${devrail_yml}" stderr="${parsed}" language=_plugins + fi + fi - local cache - cache="$(_devrail_plugin_cache_path)" + plugin_count="$(jq -r '.plugins | length' <<<"${cache_json}")" + if [[ -z "${plugin_count}" || "${plugin_count}" == "null" || "${plugin_count}" == "0" ]]; then + return 0 + fi for ((i = 0; i < plugin_count; i++)); do - name="$(yq -r ".plugins[${i}].name" "${cache}")" - cmd="$(yq -r ".plugins[${i}].targets.${target}.cmd // \"\"" "${cache}" 2>/dev/null)" + local name cmd rendered render_status final_cmd plugin_exit gate_status + name="$(IDX="${i}" jq -r '.plugins[env.IDX|tonumber].name // ""' <<<"${cache_json}")" + cmd="$(IDX="${i}" TARGET="${target}" \ + jq -r '.plugins[env.IDX|tonumber].targets[env.TARGET].cmd // ""' \ + <<<"${cache_json}")" if [[ -z "${cmd}" || "${cmd}" == "null" ]]; then - # Plugin doesn't expose this target — silent skip (no event). + # Plugin doesn't expose this target — silent skip (review L3). continue fi - if ! evaluate_gate "${i}" "${target}"; then + gate_status=0 + evaluate_gate "${cache_json}" "${i}" "${target}" "${name}" || gate_status=$? + case "${gate_status}" in + 0) ;; # gate passed; run target + 1) # gate skip — silent (event already emitted) + skipped_languages="${skipped_languages}\"${name}\"," + continue + ;; + 2) # gate config error — surface as plugin failure + overall_exit=1 + failed_languages="${failed_languages}\"${name}:gate-config\"," + if [[ "${DEVRAIL_FAIL_FAST}" == "1" ]]; then + return 1 + fi + continue + ;; + esac + + rendered="" + render_status=0 + rendered="$(render_cmd "${cache_json}" "${i}" "${target}" "${name}" "${cmd}")" || render_status=$? + if [[ "${render_status}" -ne 0 ]]; then + overall_exit=1 + failed_languages="${failed_languages}\"${name}:cmd-config\"," + if [[ "${DEVRAIL_FAIL_FAST}" == "1" ]]; then + return 1 + fi continue fi - rendered="$(render_cmd "${i}" "${target}")" - final_cmd="$(apply_override "${name}" "${target}" "${rendered}")" + final_cmd="$(apply_override "${devrail_json}" "${name}" "${target}" "${rendered}")" log_event info "plugin target executing" \ plugin="${name}" target="${target}" language=_plugins plugin_exit=0 - bash -c "${final_cmd}" || plugin_exit=$? + if [[ -n "${DEVRAIL_PLUGIN_TIMEOUT_SECONDS:-}" ]]; then + timeout -k 5 "${DEVRAIL_PLUGIN_TIMEOUT_SECONDS}" \ + bash -c "${final_cmd}" || plugin_exit=$? + else + bash -c "${final_cmd}" || plugin_exit=$? + fi if [[ "${plugin_exit}" -eq 0 ]]; then - # shellcheck disable=SC2034 # caller-scope variable from Makefile recipe ran_languages="${ran_languages}\"${name}\"," log_event info "plugin target passed" \ plugin="${name}" target="${target}" language=_plugins else - # shellcheck disable=SC2034 # caller-scope variables from Makefile recipe ran_languages="${ran_languages}\"${name}\"," - # shellcheck disable=SC2034 failed_languages="${failed_languages}\"${name}\"," - # shellcheck disable=SC2034 overall_exit=1 log_event error "plugin target failed" \ plugin="${name}" target="${target}" exit_code:="${plugin_exit}" \ diff --git a/tests/test-plugin-execution.sh b/tests/test-plugin-execution.sh index e39153a..8bc7655 100755 --- a/tests/test-plugin-execution.sh +++ b/tests/test-plugin-execution.sh @@ -1,17 +1,26 @@ #!/usr/bin/env bash # tests/test-plugin-execution.sh — Validate the plugin execution loop (Story 13.5) # -# 10 cases covering AC10: +# 15 cases covering AC10 and Story 13.5 review fixes (H1, M1, M2, M6, L2-L5): # 1. No plugins → dispatcher is a no-op (no events, no aggregation) # 2. Single plugin, lint passes → entry in ran_languages, exit 0 # 3. Single plugin, lint fails → entry in failed_languages, overall_exit=1 -# 4. Gate skip (gate path absent) → no execution, 'gate skipped' event +# 4. Gate skip (gate path absent) → no execution, 'gate skipped' event, +# plugin recorded in skipped_languages (review L5) # 5. Gate run (gate path present) → executes normally # 6. {paths} interpolation with paths_var (filtered to existing paths) # 7. Per-language override replaces manifest default cmd # 8. DEVRAIL_FAIL_FAST=1 short-circuits on first plugin failure -# 9. Manifest has 'lint' but no 'test' target → _test invocation skips silently +# 9. Manifest has 'lint' but no 'test' target → _test fully silent +# (no events at all — review L3) # 10. JSON regression: zero-plugin run produces byte-identical event shape +# 11. Absolute-path gate → config error → plugin failure (review M1) +# 12. {paths} cmd without paths_var → config error → dispatcher continues +# (review H1: helpers MUST NOT exit; recipe envelope still emits) +# 13. Path with shell-meta chars filtered before `bash -c` (review M6) +# 14. Triple-sourced lib remains a no-op (review L2: idempotency guard) +# 15. Malformed loader cache → structured error + plugin-system failure +# (review M2: yq parse errors no longer silently swallowed) # # Usage: bash tests/test-plugin-execution.sh # Env: @@ -72,8 +81,9 @@ run_dispatch_in_container() { overall_exit=0 ran_languages="" failed_languages="" + skipped_languages="" dispatch_plugin_target "'"$target"'" - echo "DISPATCH_RESULT overall_exit=$overall_exit ran_languages=${ran_languages%,} failed_languages=${failed_languages%,}" + echo "DISPATCH_RESULT overall_exit=$overall_exit ran_languages=${ran_languages%,} failed_languages=${failed_languages%,} skipped_languages=${skipped_languages%,}" exit $overall_exit ' 2>&1)" || RUN_EXIT=$? } @@ -89,8 +99,8 @@ languages: [bash] YAML run_dispatch_in_container "$WORKDIR/case1" "lint" assert_eq "0" "$RUN_EXIT" "case1 exit code" -echo "$RUN_OUT" | grep -q "DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages=" || { - echo "FAIL [case1]: expected empty ran/failed_languages, got:" >&2 +echo "$RUN_OUT" | grep -q "DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages= skipped_languages=$" || { + echo "FAIL [case1]: expected all-empty result, got:" >&2 echo "$RUN_OUT" >&2 exit 1 } @@ -119,7 +129,7 @@ languages: [elixir] YAML run_dispatch_in_container "$WORKDIR/case2" "lint" assert_eq "0" "$RUN_EXIT" "case2 exit code" -echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages="elixir" failed_languages=$' || { +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages="elixir" failed_languages= skipped_languages=$' || { echo "FAIL [case2]: expected ran_languages=\"elixir\", got:" >&2 echo "$RUN_OUT" >&2 exit 1 @@ -146,7 +156,7 @@ languages: [elixir] YAML run_dispatch_in_container "$WORKDIR/case3" "lint" assert_eq "1" "$RUN_EXIT" "case3 exit code" -echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=1 ran_languages="elixir" failed_languages="elixir"$' || { +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=1 ran_languages="elixir" failed_languages="elixir" skipped_languages=$' || { echo "FAIL [case3]: expected failed_languages=\"elixir\", got:" >&2 echo "$RUN_OUT" >&2 exit 1 @@ -174,8 +184,10 @@ languages: [elixir] YAML run_dispatch_in_container "$WORKDIR/case4" "lint" assert_eq "0" "$RUN_EXIT" "case4 exit code" -echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages=$' || { - echo "FAIL [case4]: gate skip should leave ran/failed empty, got:" >&2 +# Review L5: gate-skipped plugins now flow into skipped_languages so _test +# and _security recipes that maintain that array stay consistent. +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages= skipped_languages="elixir"$' || { + echo "FAIL [case4]: gate skip should record plugin in skipped_languages, got:" >&2 echo "$RUN_OUT" >&2 exit 1 } @@ -203,7 +215,7 @@ languages: [elixir] YAML run_dispatch_in_container "$WORKDIR/case5" "lint" assert_eq "0" "$RUN_EXIT" "case5 exit code" -echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages="elixir" failed_languages=$' || { +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages="elixir" failed_languages= skipped_languages=$' || { echo "FAIL [case5]: gate-passed lint should record ran_languages, got:" >&2 echo "$RUN_OUT" >&2 exit 1 @@ -327,14 +339,16 @@ languages: [elixir] YAML run_dispatch_in_container "$WORKDIR/case9" "test" assert_eq "0" "$RUN_EXIT" "case9 exit code (no 'test' target)" -echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages=$' || { - echo "FAIL [case9]: silent skip should leave ran/failed empty, got:" >&2 +echo "$RUN_OUT" | grep -q 'DISPATCH_RESULT overall_exit=0 ran_languages= failed_languages= skipped_languages=$' || { + echo "FAIL [case9]: silent skip should leave ran/failed/skipped empty, got:" >&2 echo "$RUN_OUT" >&2 exit 1 } -# No plugin events for the test target -if echo "$RUN_OUT" | grep -q "plugin target executing"; then - echo "FAIL [case9]: should not emit 'executing' event when target absent" >&2 +# Review L3: plugin-target absent should be GENUINELY silent — no plugin +# event of any kind for this target dispatch. +if echo "$RUN_OUT" | grep -qE 'plugin target (executing|passed|failed)|plugin gate (skipped|path)'; then + echo "FAIL [case9]: dispatcher should be fully silent when plugin doesn't expose the target" >&2 + echo "$RUN_OUT" >&2 exit 1 fi @@ -372,4 +386,182 @@ expected_shape='{"target":"lint","status":"pass","duration_ms":0,"languages":[]} got_line="$(echo "$case10_out" | grep -E '^\{.*"target":"lint"' | tail -1)" assert_eq "$expected_shape" "$got_line" "case10 JSON shape regression" -echo "==> All plugin-execution checks passed (10/10)" +# --- Case 11: absolute-path gate → config error, plugin failure --- +# Review M1: previous revision returned 1 (silent skip) for both +# gate-skip AND absolute-path config error. New contract: 2 = config +# error → plugin failure entry, not silent skip. +echo "==> Case 11: absolute-path gate rejected as config error" +mkdir -p "$WORKDIR/case11" +cat >"$WORKDIR/case11/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "true" + gates: + lint: ["/etc/passwd"] +YAML +cat >"$WORKDIR/case11/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +run_dispatch_in_container "$WORKDIR/case11" "lint" +assert_eq "1" "$RUN_EXIT" "case11 exit code (gate config error → plugin failure)" +echo "$RUN_OUT" | grep -q 'failed_languages="elixir:gate-config"' || { + echo "FAIL [case11]: gate config error must surface in failed_languages, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +echo "$RUN_OUT" >"$WORKDIR/case11.log" +assert_jq "$WORKDIR/case11.log" 'select(.level=="error" and .msg=="plugin gate path must be workspace-relative" and .plugin=="elixir")' "case11 absolute-path error event" + +# --- Case 12: {paths} cmd without paths_var → config error, plugin failure --- +# Review H1: previous revision called `exit 2` from the sourced lib, +# killing the recipe before its JSON envelope was emitted. New contract: +# render_cmd returns 2; dispatcher catches and continues (recipe still +# emits the final JSON event). +echo "==> Case 12: {paths} without paths_var → config error, dispatcher does not exit" +mkdir -p "$WORKDIR/case12" +cat >"$WORKDIR/case12/cache.yaml" <<'YAML' +plugins: + - name: alpha + rev: v1.0.0 + source: github.com/test/alpha + schema_version: 1 + targets: + lint: + cmd: "echo {paths}" + - name: beta + rev: v1.0.0 + source: github.com/test/beta + schema_version: 1 + targets: + lint: + cmd: "true" +YAML +cat >"$WORKDIR/case12/.devrail.yml" <<'YAML' +languages: [alpha, beta] +YAML +run_dispatch_in_container "$WORKDIR/case12" "lint" +assert_eq "1" "$RUN_EXIT" "case12 exit code (cmd config error → plugin failure)" +echo "$RUN_OUT" | grep -q 'failed_languages="alpha:cmd-config"' || { + echo "FAIL [case12]: cmd config error must surface in failed_languages, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +# Crucially, the SECOND plugin must still run (dispatcher returned, did +# NOT exit). Without H1's fix, the lib's `exit 2` would have killed the +# whole recipe before beta could be reached. +echo "$RUN_OUT" | grep -q 'ran_languages="beta"' || { + echo "FAIL [case12]: second plugin must still execute after first plugin's config error, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} + +# --- Case 13: path containing shell-meta chars → rejected, no injection --- +# Review M6: a path like `lib;evil` (if it existed) would inject through +# `bash -c "${final_cmd}"`. The filter rejects shell-meta characters. +echo "==> Case 13: path with shell-meta chars rejected before bash -c" +mkdir -p "$WORKDIR/case13/lib" "$WORKDIR/case13/lib;evil" +cat >"$WORKDIR/case13/cache.yaml" <<'YAML' +plugins: + - name: elixir + rev: v1.0.0 + source: github.com/community/devrail-plugin-elixir + schema_version: 1 + targets: + lint: + cmd: "echo PATHS={paths} > /workspace/result.txt" + paths_var: ELIXIR_PATHS + paths_default: "lib" +YAML +cat >"$WORKDIR/case13/.devrail.yml" <<'YAML' +languages: [elixir] +YAML +RUN_EXIT=0 +RUN_OUT="$(docker run --rm \ + -v "$WORKDIR/case13:/workspace" \ + -v "$REPO_ROOT/lib:/opt/devrail/lib:ro" \ + -e DEVRAIL_PLUGINS_CACHE=/workspace/cache.yaml \ + -e DEVRAIL_LOG_FORMAT=json \ + -e DEVRAIL_CONFIG=/workspace/.devrail.yml \ + -e ELIXIR_PATHS="lib lib;evil" \ + -w /workspace \ + "$IMAGE" \ + bash -c ' + set +e + . /opt/devrail/lib/plugin-execute.sh + overall_exit=0; ran_languages=""; failed_languages=""; skipped_languages="" + dispatch_plugin_target lint + exit $overall_exit + ' 2>&1)" || RUN_EXIT=$? +assert_eq "0" "$RUN_EXIT" "case13 exit code (filter rejects, plugin still runs with safe paths)" +result="$(cat "$WORKDIR/case13/result.txt" 2>/dev/null || true)" +# `lib;evil` must have been filtered out — only `lib` survives. +assert_eq "PATHS=lib" "$result" "case13 shell-meta path filtered before bash -c" +echo "$RUN_OUT" >"$WORKDIR/case13.log" +assert_jq "$WORKDIR/case13.log" 'select(.level=="warn" and .msg=="plugin path contains shell-meta characters; skipping" and .path=="lib;evil")' "case13 shell-meta warning event" + +# --- Case 14: double-source guard --- +# Review L2: re-sourcing the lib must be a no-op (idempotent). +echo "==> Case 14: double-source is idempotent" +mkdir -p "$WORKDIR/case14" +cat >"$WORKDIR/case14/cache.yaml" <<'YAML' +plugins: [] +YAML +cat >"$WORKDIR/case14/.devrail.yml" <<'YAML' +languages: [bash] +YAML +RUN_EXIT=0 +RUN_OUT="$(docker run --rm \ + -v "$WORKDIR/case14:/workspace" \ + -v "$REPO_ROOT/lib:/opt/devrail/lib:ro" \ + -e DEVRAIL_PLUGINS_CACHE=/workspace/cache.yaml \ + -e DEVRAIL_LOG_FORMAT=json \ + -e DEVRAIL_CONFIG=/workspace/.devrail.yml \ + -w /workspace \ + "$IMAGE" \ + bash -c ' + set +e + . /opt/devrail/lib/plugin-execute.sh + . /opt/devrail/lib/plugin-execute.sh + . /opt/devrail/lib/plugin-execute.sh + overall_exit=0; ran_languages=""; failed_languages=""; skipped_languages="" + dispatch_plugin_target lint + echo "DISPATCH_RESULT overall_exit=$overall_exit ran_languages=${ran_languages%,}" + exit $overall_exit + ' 2>&1)" || RUN_EXIT=$? +assert_eq "0" "$RUN_EXIT" "case14 exit code" +echo "$RUN_OUT" | grep -q "DISPATCH_RESULT overall_exit=0 ran_languages=$" || { + echo "FAIL [case14]: triple-sourced dispatcher must remain a no-op, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} + +# --- Case 15: malformed loader cache → recipe failure (M2) --- +# Review M2: previous revision swallowed cache parse errors via +# `2>/dev/null`. New contract: loader-cache parse failure surfaces as a +# recipe-level failure with structured error event. +echo "==> Case 15: malformed loader cache → structured error + plugin-system failure" +mkdir -p "$WORKDIR/case15" +cat >"$WORKDIR/case15/cache.yaml" <<'YAML' +plugins: + - name: broken + rev: : not-yaml ][}{ +YAML +cat >"$WORKDIR/case15/.devrail.yml" <<'YAML' +languages: [bash] +YAML +run_dispatch_in_container "$WORKDIR/case15" "lint" +assert_eq "1" "$RUN_EXIT" "case15 exit code (cache parse error)" +echo "$RUN_OUT" | grep -q 'failed_languages="_plugins:cache-parse"' || { + echo "FAIL [case15]: cache-parse error must surface in failed_languages, got:" >&2 + echo "$RUN_OUT" >&2 + exit 1 +} +echo "$RUN_OUT" >"$WORKDIR/case15.log" +assert_jq "$WORKDIR/case15.log" 'select(.level=="error" and .msg=="loader cache could not be parsed by yq")' "case15 cache-parse-error event" + +echo "==> All plugin-execution checks passed (15/15)"