What is the difference between semantic and keyword search?

Keyword search matches exact terms; semantic search uses embeddings to find conceptually similar content. "How do I return something?" would find "refund policy" via semantic search even without shared keywords.

How many results should I retrieve?

Typically 3-10 chunks work well. Fewer may miss relevant info; more adds noise and cost. Experiment with your specific use case. Re-ranking can help select the best from a larger initial set.

What is hybrid search?

Hybrid search combines keyword (BM25) and semantic (embedding) search, often getting the best of both approaches. Keywords catch exact matches; semantics catch conceptual relevance. Results are typically fused using reciprocal rank fusion.

Why does my RAG return wrong information?

Common issues: poor chunking (splitting related content), embedding model mismatch with your domain, missing relevant content in the knowledge base, or overly general queries. Debug by examining what's actually retrieved.

Book Free Assessment Calculator

Retrieval

The process of finding and fetching relevant information from a database or knowledge base in response to a query.

In-Depth Explanation

Retrieval in AI systems is the process of finding and returning relevant information from a knowledge store in response to a query. It's the "R" in RAG and a critical component of accurate AI systems.

Retrieval approaches:

Keyword search: Traditional TF-IDF or BM25 matching
Semantic search: Using embeddings to find conceptually similar content
Hybrid search: Combining keyword and semantic approaches
Knowledge graph queries: Traversing structured relationships

Retrieval pipeline stages:

Query processing: Clean and prepare the search query
Candidate selection: Find potentially relevant items efficiently
Ranking: Order candidates by relevance
Filtering: Apply business rules and metadata filters
Re-ranking: Optionally use an LLM to refine rankings

Key retrieval metrics:

Precision: Fraction of retrieved items that are relevant
Recall: Fraction of relevant items that are retrieved
MRR: How high the first relevant result appears
nDCG: Quality of ranking considering position

Effective retrieval is often the difference between a good and bad RAG system - poor retrieval produces poor answers regardless of how good your LLM is.

Business Context

Effective retrieval is the foundation of accurate AI. Poor retrieval means poor answers, regardless of how good your LLM is.

How Clever Ops Uses This

We invest heavily in retrieval optimisation for our Australian clients. Sophisticated retrieval strategies - hybrid search, re-ranking, metadata filtering - often matter more than model choice.

Example Use Case

"Retrieving the 5 most relevant document chunks to answer a customer question about your product return policy."

Frequently Asked Questions

Learn More

What is RAG (Retrieval Augmented Generation)?

Learn how RAG combines the power of large language models with your business data to provide accurate, contextual AI responses. Complete guide to understanding and implementing RAG systems.

Read article

Retool Retrospective