NVIDIA and the University of Edinburgh have evaluated agentic retrieval systems that iteratively search for evidence rather than relying on a single retrieval step. The research shows improved performance on complex retrieval benchmarks, but also substantially higher latency and token consumption. This article examines what the findings mean for enterprise AI search, RAG systems and industrial workflows.

Enterprise search has improved significantly with semantic retrieval and retrieval-augmented generation, but most systems still begin with a relatively simple assumption: a user asks a question, the system retrieves a set of relevant documents, and the model works with what it finds.

That approach works well when the information required is contained in a known document or can be located through a reasonably direct search. More complex questions are different. The first set of results may reveal new information, expose gaps in the available evidence or suggest that the original query was not specific enough.

Consider an engineer investigating repeated component failures. Relevant evidence may be distributed across design revisions, durability-test reports, inspection records, supplier documentation and earlier corrective actions. Locating those records is one part of the task. Determining what additional information should be searched for after reviewing the initial evidence is another.

Agentic AI search extends retrieval into this second stage. Instead of issuing one query and returning a fixed set of results, a language model can examine what has already been retrieved, identify missing information and initiate further searches. Retrieval therefore becomes an iterative process in which each search can inform the next.

An October 5, 2026 paper from NVIDIA and the University of Edinburgh evaluates this approach across reasoning-intensive retrieval benchmarks. The research is particularly useful because it examines not only retrieval quality but also the computational resources required to achieve it.

For enterprise AI teams, this creates a practical engineering question: whether the improvement in retrieval quality is sufficient to justify the additional latency, token consumption and infrastructure activity involved.

Why Dense Retrieval Can Miss Relevant Evidence

Dense retrieval is widely used in modern semantic search and retrieval-augmented generation systems.

It represents queries and documents as numerical embeddings and compares those representations to identify semantically similar material. The approach is efficient and has become an important foundation for retrieving candidate evidence from large document collections.

However, semantic similarity does not always capture the full relationship between a question and the information needed to answer it.

A user may describe an observed problem while the relevant document discusses its underlying mechanism using different terminology. An engineer investigating repeated equipment failures, for example, may search for a particular component while the most relevant report focuses on material properties, operating conditions or an earlier design change.

Benchmarks such as BRIGHT were created partly to evaluate these more difficult retrieval tasks, where identifying the relevant information requires reasoning rather than straightforward semantic matching.

The same limitation affects retrieval-augmented generation, or RAG. A language model can only work with the evidence supplied to it. If the retrieval stage fails to locate an important test report or engineering record, fluent answer generation cannot compensate for the missing source.

Agentic retrieval attempts to improve this stage by making search itself adaptive.

How Agentic Retrieval Works

The NVIDIA system uses a ReAct-style process in which reasoning, actions and observations are combined in an iterative loop.

The agent receives the results from an initial search and can then decide whether additional retrieval is necessary. It can formulate another query, inspect the new evidence and continue the process before returning a ranked collection of documents.

The significance lies in the fact that later searches can depend on information discovered during earlier ones.

Consider an engineering investigation in which the first set of records indicates that failures began after a supplier change. The next search could focus on supplier qualifications, material certificates and inspection findings. If the initial evidence instead points toward a revised testing procedure, the subsequent search could move toward test specifications, operating conditions and earlier validation records.

This differs from a fixed query-rewriting approach. Query rewriting improves the original search request, while an iterative agent can modify the direction of the investigation after examining retrieved evidence. The agent is selecting searches and assembling evidence; the quality of its reasoning still needs to be assessed against the documents that support its conclusions.

What the NVIDIA Benchmarks Show

The study evaluates agentic retrieval on ViDoRe v3 and BRIGHT.

ViDoRe v3 focuses on visually rich professional documents and includes tasks where relevant information may need to be collected across multiple sources. BRIGHT evaluates reasoning-intensive retrieval across domains including coding and mathematics.

For the strongest tested agent configuration using Opus 4.5, the study reports the following results:

The embedding model is matched within each benchmark comparison, although different benchmarks use different embedding models.

The results are reported using nDCG@10, or Normalized Discounted Cumulative Gain at 10. The metric measures the quality of ranked retrieval results, assigning greater value when relevant documents appear nearer the top of the first ten results.

These scores should therefore not be interpreted as answer accuracy. A score of 69.22 does not mean that 69.22% of generated answers were correct.

Retrieval quality, factual interpretation and the correctness of a final response or decision remain separate evaluation problems.

The study compares agentic retrieval primarily with standard dense retrieval. In production environments, organisations may also use hybrid retrieval, metadata filtering and reranking. Enterprise teams should therefore compare an agentic approach with the strongest practical retrieval architecture available for their own data.

The Latency and Token Cost of Deeper Search

The performance improvement comes with a significant increase in resource consumption.

On ViDoRe v3, the reported average across two tested agent configurations was approximately 107.4 seconds per query, compared with about 0.67 seconds for standard retrieval.

Average usage was approximately 764,100 input tokens and 5,800 output tokens per query.

These figures require some context. The input-token count represents cumulative consumption across the agent’s run rather than the size of a single prompt. Multiple model calls can repeatedly process instructions, retrieved evidence and earlier context.

Prompt caching can reduce some of the repeated processing cost where shared prompt prefixes are reused. Actual production expenditure will therefore depend on factors such as the model provider, pricing structure, caching behaviour and system architecture.

The latency figures also describe the experimental retrieval process rather than the complete response time of a production application. They should not be treated as a universal estimate for all agentic search systems.

Even so, the comparison demonstrates an important design consideration. More extensive retrieval can improve evidence discovery, but it may also increase latency and compute requirements considerably.

The appropriate trade-off depends on the task. A routine lookup of a known document does not require the same retrieval effort as an investigation involving multiple technical records and competing explanations.

Engineering the Retrieval Loop

NVIDIA’s engineering account also provides useful implementation details.

In one version of the system, the retriever was moved from an MCP server into the agent process, allowing a loaded retriever to be shared across tasks. Keeping the model and document embeddings resident reduced repeated loading and some deployment overhead.

This should not be interpreted as evidence that MCP is generally unsuitable for retrieval. The broader lesson is that repeated operations inside an agent loop need to be measured carefully.

Model inference, tool communication, repeated context handling and retrieval operations can all contribute to latency and cost. When an agent performs several internal actions for a single user query, relatively small inefficiencies can accumulate.

Some retrieval operations can also be executed in parallel. Independent searches can run concurrently, while searches that depend on earlier evidence must remain sequential.

This distinction becomes important when designing orchestration for production systems.

The pipeline also uses Reciprocal Rank Fusion, or RRF, as a fallback mechanism. RRF combines ranked search results based on document positions rather than requiring the underlying relevance scores to be directly comparable.

This can preserve useful evidence from searches that have already completed if an agent reaches a limit or fails to finish its entire planned sequence.

However, partial completion should remain visible to users. A returned set of documents does not necessarily mean that the investigation was exhaustive or that all relevant evidence has been identified.

Implications for Industrial AI Workflows

The NVIDIA study does not directly evaluate industrial maintenance or engineering use cases. However, the retrieval approach has clear potential applications in environments where evidence is distributed across many documents and systems.

In a maintenance investigation, an agent could search work orders, inspection records, equipment manuals and earlier failure reports. The output could assist an engineer by assembling relevant evidence, identifying contradictions and highlighting missing information.

In engineering validation, retrieval could connect a component revision with test conditions, earlier incidents and validation records.

These workflows also depend heavily on metadata quality. Equipment identifiers, revision numbers, dates, operating conditions and document versions help establish whether retrieved records belong to the same technical context.

Access controls are equally important. Sensitive documents should be filtered according to user permissions before their contents are passed to a model.

Not every enterprise question should be handled through document retrieval. Structured calculations such as downtime, production variance or inventory levels should generally rely on validated operational data and defined calculations. Narrative reports, engineering notes and investigation records require a different retrieval approach.

An enterprise AI system may therefore need to combine structured data access with semantic and agentic retrieval.

Deciding When Agentic Search Is Worth Using

The most practical conclusion from the research is not that every search query should become agentic.

The computational difference between standard retrieval and deeper iterative search suggests the need for a routing or decision layer.

A request for a known procedure or document can often be answered using conventional retrieval. A question that requires comparison across several records, reconciliation of conflicting evidence or investigation of an unclear problem may justify additional retrieval steps.

A routing mechanism could consider factors such as task complexity, evidence quality, ambiguity, available metadata, latency requirements and resource limits before selecting the appropriate retrieval strategy.

This approach would allow simple queries to remain fast while reserving more expensive agentic search for cases where additional investigation is likely to provide meaningful value.

Evaluating Agentic Search in Enterprise Systems

Enterprise evaluation should extend beyond benchmark retrieval scores.

A useful pilot should include questions that domain specialists already understand and should test both straightforward and difficult cases. It should also include scenarios in which the required evidence is incomplete or unavailable.

Relevant measures could include retrieval quality, evidence coverage, unsupported claims, completion rates, latency and total resource consumption.

For industrial applications, the ability to inspect sources is particularly important. Users need to understand which documents support an output and where uncertainty remains.

NVIDIA’s research provides useful evidence that iterative agentic retrieval can improve search performance on demanding retrieval tasks. It also shows that those improvements may come with substantially greater computational requirements.

The practical challenge for enterprise AI systems is therefore to apply deeper retrieval selectively.

Agentic search is likely to be most useful when a question requires progressive evidence gathering rather than a straightforward document lookup. The value of the approach will ultimately depend on whether the additional retrieval effort improves real-world task performance enough to justify its cost.