Similarity Search
Finding items in a database that are most similar to a query, typically using vector distance calculations on embeddings.
In-Depth Explanation
Similarity search (also called nearest neighbor search) finds items in a database that are most similar to a query vector. It's the core operation behind semantic search, recommendations, and RAG retrieval.
How similarity search works:
- Query is converted to a vector
- Algorithm searches the vector space
- Items closest to query vector are returned
- Distance/similarity scores indicate relevance
Similarity metrics:
- Cosine similarity: Angle between vectors (most common)
- Dot product: Magnitude-weighted similarity
- Euclidean distance: Straight-line distance in space
- Manhattan distance: Sum of absolute differences
Search algorithms:
- Exact (brute force): Compare against all vectors
- Approximate (ANN): Trade precision for speed
- HNSW: Hierarchical navigable small world graphs
- IVF: Inverted file indexing
Performance considerations:
- Index size and memory requirements
- Query latency vs accuracy trade-offs
- Update frequency (real-time vs batch)
Business Context
Similarity search powers product recommendations, content discovery, and the retrieval component of RAG systems.
How Clever Ops Uses This
We optimise similarity search for Australian businesses, balancing accuracy, speed, and cost for each use case.
Example Use Case
"Finding products visually similar to an item a customer is browsing, or finding documents semantically related to a query."