Severity, Confidence, and Verdict
Severity Rubric
Severity reflects consultant judgment, not scanner math. It considers task blockage, workaround difficulty, frequency, user reach, urgency, and whether the issue is a standards failure vs. best practice gap.
| Level | Meaning |
|---|---|
| Highest | A core task is blocked or severely broken for affected users |
| High | A key task is still possible but materially degraded |
| Medium | Usable with effort or workaround, but creates avoidable friction |
| Low | Minor, advisory, or best-practice-oriented |
Severity Rules
- Lower confidence does NOT automatically lower severity if impact appears serious
- Recommended improvements must not be mislabeled as fails
- Discussion items must not be escalated unless user impact is clear
- Verdict caps apply: informational notes cap at low, discussion at medium, recommendations at high
Confidence Model
Confidence is derived from evidence quality and completeness:
| Factor | High | Medium | Low |
|---|---|---|---|
| Evidence | Directly observed | Inferred | Heuristic |
| Rule type | Deterministic | Heuristic | Manual judgment |
| Signals | 3+ corroborating | 1-2 corroborating | None |
| Sources | Source + artifacts | One available | Neither |
| Platform | No ambiguity | Some ambiguity | Significant ambiguity |
Every confidence assignment includes a rationale string.
Verdict Model
Verdict is separate from severity and confidence:
| Verdict | When to use |
|---|---|
| fail | Standards violation or clearly broken accessibility behavior |
| recommended_improvement | Quality issue, best practice gap, or usable but weaker experience |
| needs_discussion | Right solution depends on product intent or unresolved tradeoffs |
| informational_note | Context or observation, not a finding |