Executive summary
Public incidents involving artificial intelligence (AI) agents can reveal failure mechanisms that a test programme does not yet cover. The new Agent Incident Registry (AIR) links 487 disclosed events to sources, stable identifiers and explicit labels. Its central lesson is methodological rather than statistical: incident shares are not failure rates. For FQI, the registry supports a controlled loop from incident intake through Corrective and Preventive Action (CAPA) to Test, Evaluation, Verification and Validation (TEVV; structured evidence of fitness for purpose) and effectiveness review.
FQI takeaway
- Incidents become test requirements: surface, trigger, authority and consequence can be mapped against existing test cases.
- Evidence remains separate from assessment: a source excerpt establishes that an event was reported; it does not prove frequency or every causal attribution.
- CAPA does not end with a fix: each relevant gap needs a reproducible test, an accountable owner and documented feedback into release and monitoring.
The principle at a glance
AIR collects reports about systems that use tools, retrieve information, exercise delegated authority or act autonomously. The authors consolidated 602 candidates into 487 records under documented merge rules. Each admitted record requires a retrievable source and evidence excerpt. The schema separates realized harm from demonstrated capability, causal role from disclosure path, and known information from explicit missingness.
Terms explained
- AI: Artificial intelligence; here, software that can plan tasks and act through tools or interfaces.
- CAPA: Corrective and Preventive Action; a structured process for correcting and preventing problems.
- TEVV: Test, Evaluation, Verification and Validation; structured evidence that an AI system is fit for its intended use.
- NIST: The US National Institute of Standards and Technology.
The full catalog contains 487 records, including 380 cases in which the agent acted. The narrower primary population contains 336 generative systems with agent action; 81 are labelled as realized harm. This 24% describes only the disclosure sample. Without counts of deployed systems and executions, it cannot establish deployment risk or support vendor comparisons.
Deep dive
That limitation helps define the registry’s quality use. AIR is not evidence of process capability; it is a structured source of failure mechanisms. It records deployment surface, trigger, causal role, consequences, safeguards and remediation. A team can ask which surfaces, permission changes and state transitions its tests represent, including destructive commands, unintended publication or faulty recovery without an attacker.
The comparison with the InjecAgent security benchmark illustrates a potential gap. In the paper’s deployment-analogue mapping, InjecAgent’s 1,054 cases occupy three of twelve surfaces and all use an attacker-controlled trigger. AIR, by contrast, contains 92 no-adversary safety failures, 87 of which are labelled as realized. This does not show that InjecAgent is a poor benchmark. It shows where additional workload, memory-state, permission, recovery and safe-response tests may be needed.
A practical FQI process begins with a versioned incident dataset. Multiple reports of the same event are merged, while uncertainty and missing information remain visible. An interdisciplinary team then maps relevant mechanisms to its own processes, tools and risks. Only a supported gap becomes a test case: defined initial state, triggering change, expected safe response, observable evidence and stop criterion. The new test runs against the actual deployed configuration and is reassessed after changes to the model, tools, permissions or infrastructure.
The National Institute of Standards and Technology (NIST) calls for documented evaluation methods, regular tracking of emerging risks and feedback from field data into measurement and risk management. The OECD framework contributes 29 common reporting criteria that jurisdictions and sectors can adapt. These sources independently support the organizational interpretation: incidents should be captured in comparable form and fed back into monitoring, risk treatment and continual improvement.
Limitations and risks
AIR is a preprint and not a representative sample. Public reporting depends on media attention, disclosure culture, language and product adoption. Thirty records rely on secondary reporting, and an excerpt confirms only that a source made a claim. One curator made the initial classifications; a second human reviewed the visible labels, so there is no independent inter-rater agreement measure. Control effectiveness is inadequately reported for 390 of 487 records. The authors also disclose that their employer sells agent-security products. Registry counts therefore support neither supplier ranking nor risk priority; prioritization remains specific to the organization and use case.
FQI conclusion
An incident registry becomes useful for quality management when it is treated as a learning and test source rather than a league table. AIR separates source, event, classification and uncertainty with unusual discipline. Its strongest transfer is the feedback loop: a supported incident exposes a possible test gap, which becomes a reproducible check in release and monitoring. Public failure experience can then support an auditable CAPA and TEVV cycle without turning into alarmism or a false risk statistic.
Sources & transparency
- Kumar et al.: The Agent Incident Registry: Toward Preventing Repeated AI Agent Failures, arXiv:2609.11030v1, submitted 10 September 2026.
- National Institute of Standards and Technology: AI Risk Management Framework Core, especially MEASURE 3.1 and 4.1–4.3 plus MANAGE 4.1–4.3.
- OECD: Towards a common reporting framework for AI incidents, 28 February 2025.
- Evidence status: registry construction, counts, benchmark comparison and limitations come from the fully reviewed preprint. The transfer to CAPA, coverage analysis, change testing and release is FQI operational interpretation supported by NIST and OECD.
- This draft was prepared with artificial-intelligence support; human subject-matter, source and publication approval are still pending.