Begin with the problem
Tables and databases contain relationships that plain text chunks can damage. Structured-data RAG preserves rows, columns, types, and executable query boundaries.
question → choose specialized retrieval path → collect multimodal/structured evidence → answerWhat you will learn
- Explain RAG with Structured Data & Tables in simple language before using its technical details.
- Follow the mechanism step by step through a small RAG example.
- Connect this topic to the modules before and after it.
- Decide when to use it, when not to use it, and what to measure in production.
Current real-system grounding: Google’s current File Search documentation includes file-based grounding and multimodal retrieval capabilities, with model, file-type, and tool-combination limitations.
The product example proves that the pattern is used in a real system. It does not mean every provider uses the same hidden algorithm, defaults, limits, or pricing.
1. The problem this module solves
Every prior module assumed knowledge belongs in a vector database. This module challenges that assumption directly — some knowledge is really better served by a traditional structured data source, and routing correctly between them is a real, practical architectural decision this course would be incomplete without.
2. The Problem — Not All Knowledge Belongs in a Vector Database
Question: "What was our total sales revenue in Q2?"
This is REALLY a precise, aggregation-style question -- it has
ONE correct, computable answer, derivable from structured records
(a sales database).
Semantic search over TEXT CHUNKS is really the WRONG tool here --
even if you had a document SUMMARIZING Q2 sales in prose, a SQL query
against the actual underlying data is more RELIABLE, more PRECISE,
and really easier to keep CURRENT than maintaining a text summary
document.3. Text-to-SQL — A Really Different RAG-Adjacent Pattern
Natural Language Question
↓
QUERY GENERATION (an LLM call, translating natural language into
SQL)
↓
SQL
↓
Execute against the DATABASE
↓
Result (precise, structured data)
↓
LLM (formats the raw result into a natural-language answer)"What was our total sales revenue in Q2?"
↓
SELECT SUM(revenue) FROM sales WHERE quarter = 'Q2'
↓
$847,320
↓
"Total sales revenue in Q2 was $847,320."Notice: this is structurally similar to RAG’s core pattern (retrieve information, then generate a natural-language answer) — but the “retrieval” step is a SQL query against structured data, not a vector similarity search against text chunks (Modules 10-18).
4. The Real Decision Framework
Is the question about a SPECIFIC, PRECISE, computable value (a sum,
count, average, or exact record lookup)?
YES -> Structured data / SQL is likely the better tool
Is the question about CONCEPTUAL, EXPLANATORY, or POLICY-type
content (what does X mean, why does Y happen, what's our approach
to Z)?
YES -> Standard RAG (Modules 10-27) is likely the better tool
Does the question REALLY need BOTH (e.g., "summarize our Q2
performance and explain what drove the change")?
-> HYBRID: structured query for the NUMBERS, RAG retrieval for
the EXPLANATORY context, combined at generation time5. Table Retrieval — When Tables Must Live IN a RAG System
Sometimes tabular data really belongs alongside unstructured content (e.g., a table embedded within a policy document, rather than a really separate database). Recall Module 6’s parsing challenge directly:
| Country | Limit |
|---------|--------|
| India | ₹5,000 |
| US | $200 |
| UK | £150 |
Naively flattening this to plain text for embedding LOSES the
explicit relationship between each country and its specific limit
(exactly Module 6's Section 5 warning).Better approach: convert EACH ROW into a CLEAR, standalone sentence
before chunking and embedding -- exactly Module 6's
"restructure_flattened_table" fix, revisited here in
the RAG-retrieval context:
"For India, the limit is ₹5,000."
"For the US, the limit is $200."
"For the UK, the limit is £150."This makes each row independently retrievable and really meaningful — directly connecting Module 6’s parsing-stage fix to this module’s retrieval-stage benefit.
6. A Real Developer Example
TechCorp's finance assistant needs to answer BOTH:
"What was our exact Q2 revenue?" -> STRUCTURED (SQL query against the
finance database) -- a precise,
computable answer
"Why did our Q2 revenue increase compared to Q1?" -> RAG (retrieve
the actual
quarterly
business review
document
explaining
what drove the
change)
"Summarize our Q2 performance" -> HYBRID: SQL query for the exact
numbers + RAG retrieval for
qualitative context, COMBINED into
one coherent, accurate summary7. A Simple Agentic AI Connection
An agent equipped with BOTH a SQL query tool and a RAG search tool directly embodies this module’s decision framework in practice — really choosing between precise structured lookups and semantic retrieval based on what a specific sub-question actually requires, exactly the kind of tool-selection reasoning your Generative AI course covered for agents generally.
8. How Is This Used in AI?
🤖 How Is This Used in AI?
Production business intelligence and analytics assistants routinely combine text-to-SQL with standard RAG retrieval — recognizing that precise numerical questions and conceptual/explanatory questions really need different underlying data sources and retrieval mechanisms, rather than forcing everything through a single, uniform pipeline.
9. Real-World Applications
- Business intelligence assistants combining precise metrics with qualitative explanation
- Customer support systems needing both exact account data (structured) and policy explanations (unstructured)
- Any application where both “give me the exact number” and “explain the context” questions really coexist
10. Common Mistakes
Incorrect idea: Forcing precise, computable questions through vector-based RAG.
Why it is incorrect: As shown directly in Section 2, this is really less reliable and harder to keep current than a direct structured query.
Incorrect idea: Naively flattening tables into plain text before embedding.
Why it is incorrect: As shown directly in Section 5 (and Module 6), this really loses important row-level relationships.
Incorrect idea: Assuming every application needs only ONE retrieval mechanism.
Why it is incorrect: As shown directly in Section 6, real applications often really need both structured and unstructured retrieval, routed appropriately per question.
11. Limitations
- Text-to-SQL generation isn’t perfectly reliable — really complex queries may require careful prompt engineering (your Prompt Engineering course) or validation before execution, especially given the real risk of a malformed or unintended query
- Combining structured and unstructured retrieval into one coherent answer adds real architectural complexity beyond either approach alone
12. Quick Reference — The Whole Idea in One Diagram
PRECISE, COMPUTABLE question -> SQL / structured
(sums, counts, exact records) data query
CONCEPTUAL, EXPLANATORY question -> Standard RAG
(policy, "why," "how") (Modules 10-27)
BOTH needed -> HYBRID: SQL for
numbers + RAG for
context, combined
Tables WITHIN unstructured docs -> restructure
each row into
a clear
sentence
BEFORE
chunking
(Module 6)13. Code — Implementing Query Routing and Table Restructuring
🎯 Target of this example: implement Section 4’s decision framework directly — routing really different question types to the appropriate retrieval mechanism, and applying Section 5’s table restructuring fix to make tabular data properly retrievable.
Example 1 — Simple
def route_query(question: str) -> str:
"""Directly implements Section 4's decision framework -- routing
based on whether a question signals a PRECISE, computable need
versus a CONCEPTUAL, explanatory need."""
structured_signals = ["total", "sum", "average", "count", "revenue", "how many", "exact"]
question_lower = question.lower()
if any(signal in question_lower for signal in structured_signals):
return "structured_data_query"
return "unstructured_rag_retrieval"
questions = [
"What was the total sales revenue in Q2?",
"What is our reimbursement policy for international travel?",
"How many employees are in the Engineering department?",
]
for q in questions:
route = route_query(q)
print(f"'{q}'\n -> {route}\n")Expected Output:
'What was the total sales revenue in Q2?'
-> structured_data_query
'What is our reimbursement policy for international travel?'
-> unstructured_rag_retrieval
'How many employees are in the Engineering department?'
-> structured_data_queryWhat we conclude from this example: really computable questions (“total,” “how many”) correctly route to structured data, while conceptual policy questions correctly route to standard RAG — exactly Section 4’s decision framework, made into a concrete routing function.
Example 2 — Intermediate
def restructure_table_row(headers: list, row_values: list) -> str:
"""Directly implements Section 5's table-restructuring fix --
converting one table row into a clear, standalone, independently
retrievable sentence, exactly reversing Module 6's flattening
problem."""
return ", ".join(f"{headers[i]}: {row_values[i]}" for i in range(len(headers)))
headers = ["Country", "Limit"]
table_rows = [
["India", "₹5,000"],
["US", "$200"],
["UK", "£150"],
]
restructured_chunks = [restructure_table_row(headers, row) for row in table_rows]
print("Restructured, independently retrievable chunks:")
for chunk in restructured_chunks:
print(f" - {chunk}")Expected Output:
Restructured, independently retrievable chunks:
- Country: India, Limit: ₹5,000
- Country: US, Limit: $200
- Country: UK, Limit: £150What we conclude from this example: each row is now an independently meaningful, retrievable statement that preserves the exact relationship between country and limit — exactly Section 5’s fix, ready to be chunked (Module 7) and embedded (Module 10) without losing the table’s real structure.
Example 3 — Production Grade
from dataclasses import dataclass
from enum import Enum
class RetrievalRoute(Enum):
STRUCTURED = "structured_data_query"
UNSTRUCTURED = "unstructured_rag_retrieval"
HYBRID = "hybrid_structured_and_unstructured"
@dataclass
class RoutingDecision:
question: str
route: RetrievalRoute
rationale: str
class HybridQueryRouter:
"""A production-style router implementing Section 4's FULL
framework, INCLUDING the hybrid case -- Section 6's real
developer example, generalized into reusable routing logic."""
STRUCTURED_SIGNALS = ["total", "sum", "average", "count", "revenue", "how many", "exact"]
EXPLANATORY_SIGNALS = ["why", "explain", "how did", "what drove"]
def route(self, question: str) -> RoutingDecision:
question_lower = question.lower()
has_structured_signal = any(s in question_lower for s in self.STRUCTURED_SIGNALS)
has_explanatory_signal = any(s in question_lower for s in self.EXPLANATORY_SIGNALS)
if has_structured_signal and has_explanatory_signal:
return RoutingDecision(
question=question, route=RetrievalRoute.HYBRID,
rationale="Needs BOTH precise numbers (SQL) AND explanatory context (RAG).")
elif has_structured_signal:
return RoutingDecision(
question=question, route=RetrievalRoute.STRUCTURED,
rationale="Precise, computable value needed -- structured query is more reliable.")
else:
return RoutingDecision(
question=question, route=RetrievalRoute.UNSTRUCTURED,
rationale="Conceptual/explanatory content -- standard RAG retrieval fits.")
router = HybridQueryRouter()
questions = [
"What was our exact Q2 revenue?",
"Why did our Q2 revenue increase compared to Q1?",
"What is our travel reimbursement policy?",
]
for q in questions:
decision = router.route(q)
print(f"'{q}'")
print(f" Route: {decision.route.value}")
print(f" Rationale: {decision.rationale}\n")Expected Output:
'What was our exact Q2 revenue?'
Route: structured_data_query
Rationale: Precise, computable value needed -- structured query is
more reliable.
'Why did our Q2 revenue increase compared to Q1?'
Route: hybrid_structured_and_unstructured
Rationale: Needs BOTH precise numbers (SQL) AND explanatory context
(RAG).
'What is our travel reimbursement policy?'
Route: unstructured_rag_retrieval
Rationale: Conceptual/explanatory content -- standard RAG retrieval
fits.What we conclude from this example: the router correctly identifies the second question as really needing BOTH structured and unstructured retrieval (it mentions “revenue” AND asks “why”), routing it to the hybrid path — exactly Section 6’s third real developer example, now implemented as reusable, automated routing logic rather than manual, case-by-case judgment.
14. Interview Questions
Q: Why is a SQL query against a structured database generally more reliable than RAG-based retrieval for answering “what was our total sales revenue in Q2?”
Ans: This question has one correct, computable answer derivable directly from structured records — a SQL aggregation query. Semantic retrieval over text chunks (even a document summarizing Q2 sales) is a really less reliable and harder-to-maintain approach, since it would depend on someone having written and kept current a prose summary, rather than querying the actual underlying data directly and precisely.
Q: Describe the text-to-SQL pattern and explain how it relates structurally to standard RAG.
Ans: Text-to-SQL translates a natural language question into a SQL query using an LLM, executes that query against a structured database, and then uses an LLM again to format the raw structured result into a natural-language answer. This is structurally similar to RAG’s core pattern — retrieve relevant information, then generate a natural- language response — but the “retrieval” step is a precise SQL query against structured data rather than a vector similarity search against text chunks.
Q: Why does naively flattening a table into plain text before embedding lose important information, and what’s the fix?
Ans: Flattening a table by simply reading cell values in sequence loses the explicit relationships between rows and columns — for example, it becomes unclear which specific limit corresponds to which specific country. The fix is restructuring each row into a clear, standalone sentence explicitly stating the relationship (like “Country: India, Limit: ₹5,000”) before chunking and embedding, so each row remains independently meaningful and retrievable without losing its original tabular relationships.
Q: Design a routing strategy for a business intelligence assistant that needs to handle both precise numerical questions and questions that really require both numbers and explanation.
Ans: I’d detect signals in the question indicating a need for precise, computable values (words like “total,” “how many,” “exact”) separately from signals indicating a need for explanatory context (words like “why,” “explain,” “what drove”). If only structured signals are present, route to a SQL query. If only explanatory signals are present, route to standard RAG retrieval. If both signal types are present — like “why did revenue increase” — route to a hybrid approach, executing both a structured query for the precise numbers and a RAG retrieval for qualitative context, then combining both results into one coherent generated answer.
15. What You Should Remember
- Precise, computable questions really belong in structured data (SQL), not vector-based RAG — verified directly through a routing function correctly distinguishing computable from conceptual questions.
- Text-to-SQL structurally mirrors RAG’s retrieve-then-generate pattern, just with a SQL query instead of vector search as the retrieval mechanism.
- Tables should be restructured into clear, standalone sentences before chunking — verified directly by converting a table into independently retrievable, relationship-preserving statements.
16. Quick Practice
For a customer support assistant handling both “what’s my current account balance” and “why was I charged this fee,” design a routing strategy using this module’s framework — which question needs structured data, which needs RAG, and does either really need both?
17. Next Step
Next: Module 31 — Multimodal RAG — closing Level 7: extending retrieval beyond text to PDFs, web content, and code repositories, each with really distinct handling requirements.
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed