Evaluate retrieval for your product

A search result can be relevant without being sufficient. Evaluate Valyu on the questions your users ask and the evidence they need to finish the job.

​

Build a small, representative test set

Start with questions from one feature. Include straightforward lookups, ambiguous names, recent events and cases where the required evidence is difficult to find. Write down what a useful result would contain before running the search. Do not use the same questions exclusively to tune and assess your configuration. Keep some examples aside to check whether a change helps beyond the queries you optimized.

​

Establish a simple baseline

Begin with the Search quickstart: five results and a short response. Save the query, settings, returned source IDs and URLs, transaction ID, cost and elapsed time. Handle sensitive user queries according to your application’s data practices. Then change one choice at a time: source scope, date window, query wording, result count or context length. A larger result set is useful only if it contains additional evidence your application needs.

​

Score evidence before scoring prose

Inspect retrieved material before sending it to your model. Otherwise a fluent answer can conceal a retrieval gap.

• Correct identity: is the result about the intended company, paper, person or product?
• Useful coverage: did it contain the facts or documents required for the task?
• Time relevance: does it cover the requested period, and is a publication date being confused with an event date?
• Usable content: is the returned text sufficient, truncated, or only an abstract?
• Traceability: can a reviewer open the source and check the claim?

Record failures separately from successful searches with no relevant results. Confirm access and endpoint support before interpreting missing specialist results as a coverage gap.

​

Measure the whole feature

Compare configurations on retrieval quality, end-to-end latency and total cost. Include subsequent Contents calls and model processing when your application uses them. For generated answers, include the time a person spends correcting the output.

​

Keep the result reproducible enough to explain

Record when you ran the evaluation and which request settings and application version you used. Live information changes, so repeating a query later is not the same as evaluating the original retrieved material. Retain the evidence needed to explain your comparison within applicable source terms. Choose the configuration that performs well on your users’ work. A single impressive example or a low per-call price is not enough to establish the best end-to-end result.

​

Compare providers on your task

Use the same queries and acceptance criteria when assessing Exa, Tavily or Parallel. These guides identify integration choices to investigate; they are not benchmark results or claims of a universal winner.