Documents vs Structured Data: What Should Your AI Agent Search?

“What is your cancellation policy?” and “What is the status of order 48172?” are both customer-support questions, but they require different kinds of information. The first calls for an explanation of a policy, including its conditions and exceptions. The second requires an exact, current record tied to a specific order.

If both questions go through the same document-search path, the system is trying to solve two different information problems with one retrieval method. This is where weak AI architecture often begins. A reliable AI agent needs to choose its source based on the kind of answer the customer is looking for, rather than treating every question as a document-search task.

A useful rule is to use documents for meaning, structured data for exact facts, live systems for current state, and combined sources when an answer requires more than one type of information. The quality of an AI answer often depends on this decision before the answer is even generated.

Start with the shape of the answer

Before choosing a retrieval method, consider what the final response needs to contain. Is the customer asking for an explanation, a specific value, or a filtered list of options? Does the answer have to reflect the current state of an account or booking? Does the question require applying a business rule to a customer-specific record?

These distinctions reveal which source is most suitable. For example:

The goal is not simply to find information that sounds related to the question. It is to retrieve the type of evidence needed to produce a correct answer. A policy explanation and an inventory lookup may both involve search, but they should not necessarily use the same search method.

Use documents when the customer needs meaning

Documents are a natural fit when a customer needs an explanation rather than a single field value. Policies, procedures, manuals, FAQs, troubleshooting guides, service descriptions, eligibility rules, and other long-form materials contain information whose meaning often spans multiple sentences or sections.

Consider questions such as “What does the warranty cover?”, “How do I check in?”, or “What happens if I miss my appointment?” The answer may depend on conditions, exceptions, and context that cannot be expressed accurately through one isolated value. In these cases, document retrieval can locate the relevant material, and the AI agent can use it to explain the answer in customer-friendly language.

Document search works particularly well when the information is primarily textual, when several passages contribute to the explanation, or when the customer needs to understand why a rule applies. It also helps when a business wants the answer to reflect its own written guidance instead of relying on the AI agent's general knowledge.

Its limitations become clearer when the customer needs an exact current number, a comparison across many rows, or a filtered set of records. A policy document is an appropriate source for explaining refund eligibility. It is usually not the right source for finding every product priced below $80 or checking an order's current delivery status.

Use structured data when the customer needs exact facts

Structured data is better suited to questions that depend on defined fields and individual records. Prices, inventory quantities, product attributes, locations, IDs, mileage, numerical factors, categories, and availability records are common examples. Such information is often stored in databases, spreadsheets, CSV files, or other tabular formats.

A customer asking “Which cars have less than 30,000 miles?” needs accurate filtering against mileage values. Someone asking “Which locations are open on Sunday?” needs the correct operating-hours fields for each location. A question about a premium factor for 110 kW requires the appropriate numeric record, not a passage that happens to mention similar values.

These are tasks where filtering, sorting, exact matching, and row-level accuracy matter. Document search may retrieve text that appears relevant without identifying the precise record the customer requested. A structured lookup can instead work directly with the fields that determine the result.

In AskHandle, Data Search serves this structured-data lookup role. It is a distinct capability from Document Search, because the problem being solved is different. Preserving the original data structure helps the workflow return consistent values and apply the right constraints.

Use live systems when the answer can change at any moment

Structured information is not necessarily current. A spreadsheet can have perfectly defined columns and still contain yesterday's information. If the customer needs to know what is true now, the AI agent may need to query the operational system that owns the latest state.

This matters for questions about order or delivery status, appointment availability, account balances, active subscriptions, payment confirmation, open support tickets, and live inventory. “Has my order shipped?” and “Do you have a 3 PM appointment today?” cannot reliably be answered from a static policy document. Even an exported spreadsheet may be too old to support the response.

Depending on the business, the authoritative source might be a CRM, booking platform, order management system, inventory service, support platform, or external API. The important distinction is not whether the source is structured, but whether it reflects the current operational state relevant to the question.

Structured does not automatically mean current. Freshness is a separate requirement, and it should be considered during source selection rather than after the answer has already been generated.

The answer format often reveals the source

One practical way to select a source is to imagine the answer before retrieving anything. The response format often signals the underlying information type.

An explanation such as “Your policy allows cancellation up to 48 hours before arrival” is likely grounded in written business knowledge. “Plan B costs $149” is a specific value that should come from a structured price record. “These three products are under $100” depends on filtering structured records, while “Your order is out for delivery” depends on the latest status in an operational system.

Some answers combine these forms. Consider: “Because your booking was made yesterday, you can still cancel without a fee.” That conclusion requires both a record containing the booking details and the policy that explains how those details affect cancellation eligibility. Neither source alone is sufficient.

This heuristic is useful because it shifts the design question from “What can the AI search?” to “What evidence must the AI retrieve to support the response?”

Do not turn structured facts into prose just to make them searchable

A common architecture mistake is to take structured information and convert it into paragraphs so that it can be processed as documents. Imagine a product catalog with 10,000 rows. Instead of looking up products directly, a team generates a text passage for every product and sends all those passages through document search.

That approach can introduce several problems. Product facts may be duplicated across generated passages, making it harder to know which version is authoritative. When a price or attribute changes, its text representation must also be updated. Products with similar descriptions may be confused during retrieval, and questions that require multiple conditions become difficult to answer precisely.

For instance, “Show me all products under $80 in size M” is fundamentally a filtering task. The same is true of comparing insurance factors, checking inventory quantities, or selecting locations by operating hours. Turning each record into prose removes much of the structure that would make those operations straightforward. It can also leave the generated text out of sync with the source data.

If the data already has useful fields and records, preserve them. Do not destroy the structure before asking AI to recover it. This principle is especially important for product catalogs, pricing tables, insurance factors, inventory lists, location directories, and large CSV or spreadsheet sources.

Do not use structured lookup for questions that require explanation

The reverse mistake is also possible. A database may contain the fields cancellation_window = 48 and cancellation_fee = 20, but those values do not fully answer a customer who asks, “Why am I being charged a cancellation fee?”

The customer may need the language of the cancellation policy, an explanation of timing rules, any applicable exceptions, and the context of their own booking. The numeric values are important, but they do not communicate the complete reasoning behind the fee.

Structured data supplies precise facts. Documents often supply the meaning and conditions surrounding those facts. Choosing a source should therefore depend on what the question demands, not only on the format in which some related information happens to be stored.

Mixed-source questions are common

Many customer-support requests need more than one source because they ask the AI agent to apply a rule to a particular person, purchase, or situation. The agent has to retrieve the facts, retrieve the relevant guidance, and determine how they fit together.

For a cancellation request such as “Can I cancel this booking without a fee?”, the workflow may need the booking date, current booking status, and cancellation policy. To answer “Is this product still covered?”, it may need a purchase date, product model, and warranty terms. An estimated insurance premium may depend on customer inputs, a factor table, calculation logic, and a policy disclaimer. A late-delivery question may require the current order status and the company's delivery policy.

The common pattern is:

Structured or live record → Relevant policy or document → Rule application or calculation → Customer answer

In some cases, more than two sources are needed, or the workflow must clarify a missing detail before continuing. The value of a mixed-source workflow is that it brings exact, situation-specific facts together with the explanatory context needed to produce a useful answer.

Source selection can happen before or during the workflow

There are two common approaches to source selection. In a pre-routed workflow, the system already knows which source to use for a category of question. Inventory questions go to Data Search, policy questions go to Document Search, and order-status questions go to a connected operational system. This arrangement can be simple and predictable when the question categories are well defined.

A dynamic workflow decides which source or sources to use as the request is processed. Suppose a customer asks, “Can I cancel?” The AI agent might first retrieve the cancellation policy, then recognize that the answer also depends on the customer's booking date or status. It can then obtain the booking record before applying the rule.

Both approaches have value. The appropriate design depends on how varied the requests are and whether the required source can be determined at the beginning of the conversation. In AskHandle, this routing can involve the Router, Dispatcher, AI Answer, Document Search, and Data Search nodes, with connected systems used when the relevant information resides outside uploaded knowledge or data sources.

The key is to direct each question through a path suited to the information it needs, including a second source when the first is not enough.

What happens when the wrong source is used?

Source-selection errors can look like AI hallucinations even when the immediate problem is that the workflow retrieved the wrong evidence. The response may be fluent and confident, while the underlying information is outdated, incomplete, or associated with the wrong record.

For example, if a customer asks for the price of Plan B and the AI agent consults an old marketing brochure instead of the current price table, it may give a believable but incorrect amount. If a customer asks why a policy applies and the system returns only a database code, the underlying value may be correct while the explanation remains incomplete.

The same failure can occur when the source is stale. An agent checking yesterday's spreadsheet export cannot reliably confirm whether a room is available tonight. More subtly, the agent might retrieve the correct cancellation policy but pair it with someone else's booking. The policy itself is accurate, yet the customer-specific conclusion is wrong.

These problems are not necessarily fixed by rewriting the AI prompt. They require attention to source selection, freshness, record matching, and workflow design. Better wording cannot compensate for evidence that does not support the answer.

A practical source-selection table

The following table provides a quick reference when designing a customer-facing workflow.

The table is a starting point, not a replacement for understanding the question. For instance, a general question about a cancellation policy is different from a request to determine whether a specific booking qualifies for a free cancellation. The latter usually needs both the policy and the booking record.

How this maps to AskHandle

In AskHandle, source selection can be part of the AI agent's workflow rather than an assumption that every request should search the same knowledge collection. A customer message can be routed through the Router or Dispatcher to an appropriate path, such as AI Answer with available business knowledge, Document Search for explanatory material, Data Search for structured records, or a connected system for current operational information.

If the question needs multiple sources, the workflow can bring the approved results together before composing the answer. When the available information cannot support a reliable response, the agent can ask a clarifying question or move to Human Handoff, depending on the workflow configuration.

This distinction is straightforward in common requests. “What is your cancellation policy?” points toward Document Search. “Which options are under $100?” points toward Data Search. “Is my current booking still active?” requires access to the system that owns the booking state. “Can I cancel my active booking without a fee?” needs both the live booking record and the policy explaining the cancellation terms.

The purpose is not to make the workflow complex for its own sake. It is to ensure that the customer receives an answer grounded in the right kind of information, with the right level of accuracy and context.

Choose the source that owns the fact

A reliable customer-facing AI agent should not use one retrieval method for every question. Documents are strongest when the customer needs meaning and explanation. Structured data is suited to exact facts and filtered results. Live systems are necessary when the answer depends on current state, while mixed-source workflows help apply rules to specific records and circumstances.

When each question is directed to the source that naturally owns the required information, the workflow is better positioned to return accurate, useful answers. The best AI answer often begins before generation, with the decision about which facts to retrieve and where to retrieve them.

Source-selection questions

Should an AI agent use document search for every question?

No. Document search is useful when the answer depends on policies, procedures, explanations, or other textual knowledge. Exact records, filters, numerical values, and live account state are often better handled through structured data or connected systems.

When should AI use structured data instead of documents?

Use structured data when the answer depends on exact fields, filtering, sorting, numeric values, catalogs, or row-level records. This allows the workflow to work with the original structure instead of searching through prose that describes the data.

Can an AI agent search spreadsheets?

Yes. Spreadsheets and CSV files can contain valuable structured information, including prices, inventory, factors, and product details. Where possible, the workflow should preserve that structure rather than converting everything into text.

When does AI need live data?

AI needs live data when the answer depends on current operational or account state, such as order status, appointment availability, subscription state, payment confirmation, or active inventory. A structured export alone may not be recent enough.

Can AI combine documents and structured data?

Yes. Many support questions require both. A cancellation request, for example, may need a booking record and the relevant cancellation policy before the AI agent can determine whether a fee applies.