Implement efficient similarity search with vector databases

Similarity search patterns

by wshobson · MIT license · ★ 39,857 on GitHub · updated Sep 21

Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance.

---
name: similarity-search-patterns
description: Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance.
---

# Similarity Search Patterns

Patterns for implementing efficient similarity search in production systems.

## When to Use This Skill

- Building semantic search systems
- Implementing RAG retrieval
- Creating recommendation engines
- Optimizing search latency
- Scaling to millions of vectors
- Combining semantic and keyword search

Overview

Patterns for implementing efficient similarity search in production systems.

Manual install (npx, ZIP)

Claude Codenpx skills add wshobson/agents --skill similarity-search-patterns -g -a claude-code

Claude Code (manual)

mkdir -p ~/.claude/skills && curl -sL https://codeload.github.com/wshobson/agents/tar.gz/4236bb91f839 \
  | tar -xz -C ~/.claude/skills --strip-components=4 agents-4236bb91f839/plugins/llm-application-dev/skills/similarity-search-patterns

Windows (PowerShell)

New-Item -ItemType Directory -Force "$HOME\.claude\skills" | Out-Null; Invoke-WebRequest "https://m.agentalley.io/zip/8698954798a30abe0a77/similarity-search-patterns.zip" -OutFile "$env:TEMP\similarity-search-patterns.zip"; Expand-Archive "$env:TEMP\similarity-search-patterns.zip" -DestinationPath "$HOME\.claude\skills" -Force

Claude.ai / desktopDownload the ZIP and upload it under Settings → Capabilities → Skills.

Codex, Cursor, other agentsPoint them at SKILL.md. The other files are plain text.

Connects with Agent Alley MCP · agentalley.io/mcp

Files included

SKILL.md2.7 KB
references/1 file · 15.0 KB

Read the files here

wshobson/4236bb92 files

SKILL.md2.7 KB

SKILL.md

Show the full text65 lines

similarity-search-patterns/SKILL.md65 lines · 2.7 KB

PreviewCode

Outline

RawView on GitHub

Similarity Search Patterns

Patterns for implementing efficient similarity search in production systems.

When to Use This Skill

Building semantic search systems
Implementing RAG retrieval
Creating recommendation engines
Optimizing search latency
Scaling to millions of vectors
Combining semantic and keyword search

Core Concepts

1. Distance Metrics

| Metric | Formula | Best For | | ------------------ | ------------------ | --------------------- | --- | -------------- | | Cosine | 1 - (A·B)/(‖A‖‖B‖) | Normalized embeddings | | Euclidean (L2) | √Σ(a-b)² | Raw embeddings | | Dot Product | A·B | Magnitude matters | | Manhattan (L1) | Σ | a-b | | Sparse vectors |

2. Index Types

┌─────────────────────────────────────────────────┐
│                 Index Types                      │
├─────────────┬───────────────┬───────────────────┤
│    Flat     │     HNSW      │    IVF+PQ         │
│ (Exact)     │ (Graph-based) │ (Quantized)       │
├─────────────┼───────────────┼───────────────────┤
│ O(n) search │ O(log n)      │ O(√n)             │
│ 100% recall │ ~95-99%       │ ~90-95%           │
│ Small data  │ Medium-Large  │ Very Large        │
└─────────────┴───────────────┴───────────────────┘

Templates and detailed worked examples

Full template library and detailed worked examples live in references/details.md. Read that file when you need the concrete templates.

Best Practices

Do's

Use appropriate index - HNSW for most cases
Tune parameters - ef_search, nprobe for recall/speed
Implement hybrid search - Combine with keyword search
Monitor recall - Measure search quality
Pre-filter when possible - Reduce search space

Don'ts

Don't skip evaluation - Measure before optimizing
Don't over-index - Start with flat, scale up
Don't ignore latency - P99 matters for UX
Don't forget costs - Vector storage adds up

1	---
2	name: similarity-search-patterns
3	description: Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance.
4	---
5
6	# Similarity Search Patterns
7
8	Patterns for implementing efficient similarity search in production systems.
9
10	## When to Use This Skill
11
12	- Building semantic search systems
13	- Implementing RAG retrieval
14	- Creating recommendation engines
15	- Optimizing search latency
16	- Scaling to millions of vectors
17	- Combining semantic and keyword search
18
19	## Core Concepts
20
21	### 1. Distance Metrics
22
23	\| Metric \| Formula \| Best For \|
24	\| ------------------ \| ------------------ \| --------------------- \| --- \| -------------- \|
25	\| Cosine \| 1 - (A·B)/(‖A‖‖B‖) \| Normalized embeddings \|
26	\| Euclidean (L2) \| √Σ(a-b)² \| Raw embeddings \|
27	\| Dot Product \| A·B \| Magnitude matters \|
28	\| Manhattan (L1) \| Σ \| a-b \| \| Sparse vectors \|
29
30	### 2. Index Types
31
32	```
33	┌─────────────────────────────────────────────────┐
34	│ Index Types │
35	├─────────────┬───────────────┬───────────────────┤
36	│ Flat │ HNSW │ IVF+PQ │
37	│ (Exact) │ (Graph-based) │ (Quantized) │
38	├─────────────┼───────────────┼───────────────────┤
39	│ O(n) search │ O(log n) │ O(√n) │
40	│ 100% recall │ ~95-99% │ ~90-95% │
41	│ Small data │ Medium-Large │ Very Large │
42	└─────────────┴───────────────┴───────────────────┘
43	```
44
45	## Templates and detailed worked examples
46
47	Full template library and detailed worked examples live in `references/details.md`. Read that file when you need the concrete templates.
48
49	## Best Practices
50
51	### Do's
52
53	- Use appropriate index - HNSW for most cases
54	- Tune parameters - ef_search, nprobe for recall/speed
55	- Implement hybrid search - Combine with keyword search
56	- Monitor recall - Measure search quality
57	- Pre-filter when possible - Reduce search space
58
59	### Don'ts
60
61	- Don't skip evaluation - Measure before optimizing
62	- Don't over-index - Start with flat, scale up
63	- Don't ignore latency - P99 matters for UX
64	- Don't forget costs - Vector storage adds up
65