Foxley Quality Intelligence · Research Brief

What an AI agent credential actually proves

A signed agent credential applies to a tested configuration, task, scoring rule and validity period—not to every use of the agent.

By Johann FischerPublished 21 September 2026Original reviewed 21 September 2026Translation reviewed 21 September 2026

Supplier qualityLieferantenauditKonfigurationsmanagementLieferantenqualifikationAudit

4 min read

Four-stage review path: identify the agent configuration and test scope, assess evidence, verify the signed record, then make a human fitness-for-purpose decision. — FQI original illustration: a valid signature does not replace expert review of the measurement and its use boundary. Based on Drew and Zhou, W3C and NIST.

Executive summary

Artificial intelligence (AI) can power an agent. A performance claim for that agent must not be read as a universal seal of quality. The non-peer-reviewed LEGIT preprint proposes a signed digital credential that binds measured quality and cost to a particular configuration, task domain, test set, resource limit, evidence and validity period. The key FQI boundary is that a valid signature confirms the provenance and integrity of the record. It does not, by itself, establish that the measurement is correct or that the agent is fit for a buyer’s intended use.

FQI takeaway

Qualify only the tested configuration: The model, surrounding control software, tools, parameters, task domain and budget jointly define the object of evaluation.
Separate evidence integrity from truth: A signature and cryptographic fingerprint protect provenance and integrity; methods, reference answers and scope still need expert assessment.
Monitor validity: Configuration changes, expiry, suspension or new use conditions call for reassessment instead of silently reusing an old score.

The principle at a glance

LEGIT has three layers: initial capability evaluation, separate records of later task outcomes, and proposed marketplace allocation. Its digital record, called a credential, identifies the agent configuration, tested domains, task set or sampling record, scoring rule, token budget (a limit measured in units of text), quality, cost per solved task, evidence references, issuer, evaluation date and expiry. Reputation based on later operation is separate from this static evaluation record.

Terms explained

AI: Artificial intelligence; systems that derive patterns from data or support tasks automatically.
Agent harness: The technical control environment managing instructions, planning, tool calls, memory and stop rules around a model.
Credential: A structured digital record of an issuer’s claims about a precisely identified subject.
Hash: A cryptographic fingerprint that makes changes to a particular data version detectable.
Token: A unit of text processed by the model; a token budget limits the amount processed and part of its resource use.
W3C: World Wide Web Consortium, a body developing Web standards.
NIST: The US National Institute of Standards and Technology.

Deep dive

The authors define an agent as more than its model. The evaluated object includes model dependencies, the agent harness, available tools and parameters. A model-version change, different tool permissions or revised context management can therefore create a new configuration. The proposed credential also distinguishes properties observed by the evaluator from vendor declarations and dependencies that remain unknown for opaque services.

The study compares 18 configurations using three models from two vendors and selected harness variants. Benchmark comparisons generally use 127 shared tasks under specified token budgets, drawing on GAIA and MATH-500 tasks. Outcomes include solved tasks and cost per solved task. Corrected statistical comparisons support differences between the tested models, while quality contrasts among the tested harnesses remain inconclusive. Within-model cost differences are visible. These findings motivate recording quality, cost, budget and configuration together; they do not prove the benefit of the complete marketplace mechanism.

For supplier qualification, the FQI transfer is a controlled evaluation plan. Before procurement, an organization defines its target task population, minimum quality, cost and time limits, critical failure types, reference judgments and repeat runs. The supplier provides more than an overall score: a configuration identifier, test version, item-level results or inspectable evidence, measurement date and known dependencies. Approval applies only to the documented use. A new model version or different tool permissions trigger change assessment.

The World Wide Web Consortium (W3C) explicitly separates technical verification from business validation of verifiable credentials. An authentic, current credential is a statement by its issuer; it does not establish that a claim is true or suitable for a particular purpose. A verifier must assess issuer, proof, subject and claims against its own policy. The NIST AI Risk Management Framework likewise calls for measurements and limitations to be documented and monitored in the relevant context of use.

Limitations and risks

LEGIT is a preprint, not an established certification standard. Its experiments cover selected single-model configurations and two task pools; dynamic model mixtures, delegated agents and other domains were not evaluated. The end-to-end effect of its proposed marketplace allocation was not compared with simpler selection methods. A signature cannot reveal an undisclosed change to an opaque service. Reputation also depends on the quality and resistance to manipulation of reported task outcomes. Without a trusted issuer, suitable reference evidence and a reassessment rule, a technically valid credential can create false assurance.

FQI conclusion

An agent credential is useful when it narrows a performance claim rather than broadening it. It should show who measured what, on which configuration, tasks, rules and resources, and for how long that evidence remains applicable. The deployment decision stays with the user: verify the signature, assess the evidence, compare the use boundary and monitor changes. That turns a score into an auditable qualification record without treating it as a universal quality seal.

Sources & transparency

Drew and Zhou: LEGIT: Credentialing Protocol for Trustworthy AI Agent Marketplaces, arXiv:2609.21325v1, submitted 18 September 2026; full HTML text.
World Wide Web Consortium: Verifiable Credentials Data Model v2.0, W3C Recommendation of 15 May 2025.
National Institute of Standards and Technology: AI Risk Management Framework Core.

Editorial note: LEGIT’s design and experimental results come from the non-peer-reviewed preprint. Applying them to supplier qualification, change control and release decisions is FQI interpretation. The proposal is not an accredited conformity assessment. This AI-assisted text remains a draft until explicit human approval.

What is this about?

Artificial intelligence (AI) can power an agent that uses a language model and additional tools to perform tasks. A buyer wants to know whether such a service will handle its own work reliably. One performance score cannot answer that question by itself. Without further details, it does not identify the tested version, tasks, tools or limits.

In the non-peer-reviewed LEGIT preprint, researchers Drew and Zhou propose a digital performance credential. It records the specific agent configuration, task domain, scoring rule, resource limit, measured quality and cost, links to evaluation evidence, issuer, date and expiry. A digital signature is intended to show who issued the record and whether its signed contents changed.

Why does it matter?

An agent is more than its language model. The software around the model controls instructions, tool calls and memory, among other things. If a supplier changes one of these parts, behavior may change even though the product name stays the same. Approval should therefore apply to the configuration that was actually tested.

The authors compared 18 selected configurations using three models from two vendors. Individual benchmark comparisons generally used 127 shared tasks under a fixed token budget, a limit measured in units of text. Such a budget limits how much text the system can process and therefore part of its resource use. The study found more conclusive quality differences between the tested models than between the tested variants of the surrounding control software. Cost differences within a model were still visible. This does not prove that the proposed marketplace produces better buying decisions: the study did not test that end-to-end outcome.

A simple example

Simplified example: A company seeks an agent to sort incoming supplier reports. Supplier A advertises “90% correct”. The quality team asks: On which reports? Against which expert-reviewed answers? With what model version and tool permissions? How often was testing repeated? What did a correct completion cost?

The supplier provides a signed record with those answers and an expiry date. The team can now check whether the record really came from the named issuer and has remained unchanged. That still does not justify release. Specialists must compare the test cases with their own reports, examine critical errors and define when a person makes the decision. A later model or tool-permission change requires a fresh assessment of that configuration.

What should you watch out for?

The W3C specification for verifiable credentials separates two questions. Technical verification can establish that a credential is authentic and current. Whether its claims are true and fit the intended use requires additional assessment. A signature cannot turn a poor measurement into a good one.

The proposed LEGIT credential also has limits. The paper has not been peer-reviewed and tested selected single-model configurations on two task pools. It did not test whether mixtures of models, delegated agents or the full marketplace work better as a result. With an opaque service, a buyer might not see an internal change. Later user feedback can also be incomplete or manipulated. The National Institute of Standards and Technology (NIST) calls for measurement and monitoring in the actual context of use in its AI risk framework.

What should you remember?

A performance credential applies to the documented configuration and tested tasks, not to every possible use.
A valid signature establishes the record’s provenance, not the truth or fitness of its claims.
Before release, people must assess the evidence, use boundary, changes and expiry rules.

Sources & transparency

Drew and Zhou: LEGIT: Credentialing Protocol for Trustworthy AI Agent Marketplaces, arXiv:2609.21325v1.
World Wide Web Consortium: Verifiable Credentials Data Model v2.0.
National Institute of Standards and Technology: AI Risk Management Framework Core.

Figures and study limits come from the Research Brief. The supplier scenario is simplified and the release rules are FQI interpretation. This remains a draft without human approval.

Categories & methods

LieferantenauditKonfigurationsmanagementLieferantenqualifikationAuditFreigabeSupplier qualityAI & automationAI agents

Executive summary

FQI takeaway

The principle at a glance

Terms explained

Deep dive

Limitations and risks

FQI conclusion

Sources & transparency

What is this about?

Why does it matter?

A simple example

What should you watch out for?

What should you remember?

Sources & transparency

Categories & methods

Similar reports

Benchmarks Passed, Operations Unknown: Qualifying LLM Services with Incident Metadata

Explanation under counter-test: remove hidden AI influences before release

Check before agent action: one shared release boundary