Test the flows
BMad Method tests skills with evals.json (a prompt, fixture files, and a
grading rubric) and triggers.json (does the skill fire for a given question).
A live run uses bmad-eval-runner.
BGlad keeps that file shape under evals/ at the repo root for a later live
pass. CI does not call a model. It runs flow scenarios: JSON playthroughs
checked against the help catalog and skill text.
python3 -m unittest tests.test_flow_scenarios| Layer | What it proves |
|---|---|
| Contract tests | Menus, overlays, docs pages |
| Flow scenarios | Dev vs Analysis sequence, when the agent must ask or stop, skip stock BMad |
| Evals (later) | A live model actually follows a scenario |
Scenarios live in tests/flow_scenarios/scenarios/. The toy ticket is
DEMO-42 (docs homepage names Dev and Analysis). dev-halt-open-questions
and analysis-halt-open-questions are proposals that must stop with Open
questions. Swap the fixture when you have a real issue to grade.
Live evals still need a non-interactive (“headless”) mode on the skills, and a
Jira / GitLab / GitHub mock or a throwaway project. Until then,
evals/*/evals.json is the rubric.