Executive summary
The US National Institute of Standards and Technology has published TEVV-Athlon, a four-stage draft framework for evaluating AI systems. TEVV—test, evaluation, verification and validation—is adapted to the intended use context. For quality management, the relevant object is therefore not just a model score, but the complete application system with its objectives, risks, data, tools and real-world effects.
FQI takeaway
- AI validation must be use-case specific: A general model score cannot replace testing against process requirements and failure risks.
- Failure Mode and Effects Analysis (FMEA) and TEVV complement each other: FMEA describes what can go wrong; a TEVV plan produces evidence that prevention and detection controls actually work.
- Release needs several forms of evidence: Performance, robustness, safety, traceability and operating limits must be tested separately.
The principle at a glance
National Institute of Standards and Technology (NIST) published the initial public draft of NIST AI 200-2 on 7 August 2026. The comment period runs until 6 October 2026. The framework addresses not only conventional machine-learning models but also large language models, multimodal systems and agentic AI. It is an official draft, not yet a binding standard. Its strength therefore lies less in normative force than in a structured evaluation logic that can already be transferred to quality and release processes.
Terms explained
- AI: Artificial intelligence; systems that derive patterns from data or support tasks automatically.
- NIST: The US National Institute of Standards and Technology.
- FMEA: Failure Mode and Effects Analysis; a method for proactively assessing risks.
- 8D: Eight Disciplines problem-solving method; D1 through D8 structure the team, problem, cause, actions and closure.
- TEVV: Test, Evaluation, Verification and Validation; systematic evidence that requirements and intended use are satisfied.
- prompt: A task instruction or input for an AI system; it describes the task, rules, context and expected result.
Deep dive
TEVV stands for test, evaluation, verification and validation. These terms are often mixed in practice, but their separation matters in a reliable quality system. Testing creates observations under defined conditions. Evaluation judges those results against criteria. Verification checks whether specified requirements were implemented. Validation determines whether the system fulfils its purpose in the intended use context.
TEVV-Athlon organises evaluation into four stages: Articulate & Organize, Define & Construct, Apply & Measure, and Synthesize & Interrogate. In simplified terms: establish objectives and the system boundary, construct the measurement concept and test events, conduct the tests, and interpret the results in context. The final step is particularly important. A single mean value or global accuracy score can conceal local risks such as rare wrong decisions with severe consequences.
For quality management, this directly transfers to test planning. Before testing, the organisation must determine which quality characteristics matter, who is affected, which failure scenarios are acceptable and what evidence is sufficient for release. An AI system classifying complaints needs different acceptance criteria from one that releases inspection instructions or automatically changes process parameters. Technical model performance is only one part of the overall assessment.
The connection to FMEA is especially direct. A system or process FMEA can identify failure modes such as incorrect classification, an incomplete source base, unauthorised tool use, delayed escalation or undetected data drift. Specific TEVV test events are then derived from those failure modes. This creates a closed chain: risk identification, test requirement, test result, evaluation and release decision. FMEA actions are not merely documented; their effectiveness is tested.
The approach also adds value to audits. The auditable object is not a claim that AI “works well”, but the traceable evidence chain: requirement, test case, dataset, model and agent-system version, result, deviation, assessment and decision. This resembles conventional inspection-equipment and software validation, while extending them to stochastic results, dynamic data and context-dependent behaviour.
A qualified baseline should be established before release. Every later change to the model, prompt, knowledge base, tool or interface is tested against this reference. TEVV thereby links initial validation with controlled change management and periodic revalidation during operation.
Two applications are particularly apparent. First, TEVV-Athlon can structure the release of an AI assistant in an 8D process. Test cases would cover not only linguistic quality, but also source grounding, correct escalation, handling of missing facts and separation of causes from assumptions. Second, a visual-defect detection system can be assessed across defined defect classes, boundary samples, lighting variants, unknown parts and drift scenarios. Alongside sensitivity and specificity, repeatability, false-alarm cost and reaction plans should be tested.
Limitations and risks
First, TEVV-Athlon is a draft. Terms, examples and requirements may change after consultation. Organisations should use it as methodological guidance, not as a certification requirement.
Second, a structured framework does not guarantee good testing. Quality still depends on representative data, suitable metrics and realistic failure scenarios. A formally complete test plan can create false confidence when critical boundary conditions are absent.
Third, there is a risk of metric tunnel vision. Optimising only easily measured model metrics can overlook process risks, user behaviour and downstream effects. Agentic systems also require tools, permissions, stopping conditions and external systems to be included in the evaluation.
Fourth, the effort can be substantial. Maximum validation depth would be uneconomic for low-risk assistance functions. Evaluation depth must scale with risk.
FQI conclusion
TEVV-Athlon does not give quality management a finished checklist, but it offers a reliable way of thinking. The decisive shift is that release applies not to an isolated model, but to an AI-assisted work system in a defined use context. FMEA, test planning, audit and release can thereby form a consistent evidence chain. Organisations moving AI from pilot status into quality-relevant processes need precisely this system perspective.
Sources & transparency
- National Institute of Standards and Technology: The TEVV-Athlon Framework for Evaluating AI Systems, announcement dated 7 August 2026.
- Evidence status: official initial public draft; not yet a final standard or basis for certification.
- FQI transfer: Linking the framework to FMEA, test planning, audit and release is a professional inference by FQI, not an explicit requirement of the NIST document.
- This article was researched and written with AI support. Sources and professional transfers were pre-reviewed with AI. Human professional and source approval was documented on 10 August 2026, and the article is approved for publication.
- Approval status: First-use explanations and the glossary were reviewed by a human and approved for publication on 17 August 2026. Approved by: Johann Fischer.