Select and optimize embedding models for semantic search

Embedding strategies

by wshobson · MIT license · ★ 39,857 on GitHub · updated Sep 21

Select and optimize embedding models for semantic search and RAG applications. Use when choosing embedding models, implementing chunking strategies, or optimizing embedding quality for specific domains.

---
name: embedding-strategies
description: Select and optimize embedding models for semantic search and RAG applications. Use when choosing embedding models, implementing chunking strategies, or optimizing embedding quality for specific domains.
---

# Embedding Strategies

Guide to selecting and optimizing embedding models for vector search applications.

## When to Use This Skill

- Choosing embedding models for RAG
- Optimizing chunking strategies
- Fine-tuning embeddings for domains
- Comparing embedding model performance
- Reducing embedding dimensions
- Handling multilingual content

Overview

Guide to selecting and optimizing embedding models for vector search applications.

Manual install (npx, ZIP)

Claude Codenpx skills add wshobson/agents --skill embedding-strategies -g -a claude-code

Claude Code (manual)

mkdir -p ~/.claude/skills && curl -sL https://codeload.github.com/wshobson/agents/tar.gz/4236bb91f839 \
  | tar -xz -C ~/.claude/skills --strip-components=4 agents-4236bb91f839/plugins/llm-application-dev/skills/embedding-strategies

Windows (PowerShell)

New-Item -ItemType Directory -Force "$HOME\.claude\skills" | Out-Null; Invoke-WebRequest "https://m.agentalley.io/zip/bce4027e954f3226f55d/embedding-strategies.zip" -OutFile "$env:TEMP\embedding-strategies.zip"; Expand-Archive "$env:TEMP\embedding-strategies.zip" -DestinationPath "$HOME\.claude\skills" -Force

Claude.ai / desktopDownload the ZIP and upload it under Settings → Capabilities → Skills.

Codex, Cursor, other agentsPoint them at SKILL.md. The other files are plain text.

Connects with Agent Alley MCP · agentalley.io/mcp

Files included

SKILL.md2.8 KB
references/1 file · 16.4 KB

Read the files here

wshobson/4236bb92 files

SKILL.md2.8 KB

SKILL.md

Show the full text66 lines

embedding-strategies/SKILL.md66 lines · 2.8 KB

PreviewCode

Outline

RawView on GitHub

Embedding Strategies

Guide to selecting and optimizing embedding models for vector search applications.

When to Use This Skill

Choosing embedding models for RAG
Optimizing chunking strategies
Fine-tuning embeddings for domains
Comparing embedding model performance
Reducing embedding dimensions
Handling multilingual content

Core Concepts

1. Embedding Model Comparison (2026)

Model	Dimensions	Max Tokens	Best For
voyage-3-large	1024	32000	Claude apps (Anthropic recommended)
voyage-3	1024	32000	Claude apps, cost-effective
voyage-code-3	1024	32000	Code search
voyage-finance-2	1024	32000	Financial documents
voyage-law-2	1024	32000	Legal documents
text-embedding-3-large	3072	8191	OpenAI apps, high accuracy
text-embedding-3-small	1536	8191	OpenAI apps, cost-effective
bge-large-en-v1.5	1024	512	Open source, local deployment
all-MiniLM-L6-v2	384	256	Fast, lightweight
multilingual-e5-large	1024	512	Multi-language

2. Embedding Pipeline

Document → Chunking → Preprocessing → Embedding Model → Vector
                ↓
        [Overlap, Size]  [Clean, Normalize]  [API/Local]

Templates and detailed worked examples

Full template library and detailed worked examples live in references/details.md. Read that file when you need the concrete templates.

Best Practices

Do's

Match model to use case: Code vs prose vs multilingual
Chunk thoughtfully: Preserve semantic boundaries
Normalize embeddings: For cosine similarity search
Batch requests: More efficient than one-by-one
Cache embeddings: Avoid recomputing for static content
Use Voyage AI for Claude apps: Recommended by Anthropic

Don'ts

Don't ignore token limits: Truncation loses information
Don't mix embedding models: Incompatible vector spaces
Don't skip preprocessing: Garbage in, garbage out
Don't over-chunk: Lose important context
Don't forget metadata: Essential for filtering and debugging

1	---
2	name: embedding-strategies
3	description: Select and optimize embedding models for semantic search and RAG applications. Use when choosing embedding models, implementing chunking strategies, or optimizing embedding quality for specific domains.
4	---
5
6	# Embedding Strategies
7
8	Guide to selecting and optimizing embedding models for vector search applications.
9
10	## When to Use This Skill
11
12	- Choosing embedding models for RAG
13	- Optimizing chunking strategies
14	- Fine-tuning embeddings for domains
15	- Comparing embedding model performance
16	- Reducing embedding dimensions
17	- Handling multilingual content
18
19	## Core Concepts
20
21	### 1. Embedding Model Comparison (2026)
22
23	\| Model \| Dimensions \| Max Tokens \| Best For \|
24	\| -------------------------- \| ---------- \| ---------- \| ----------------------------------- \|
25	\| voyage-3-large \| 1024 \| 32000 \| Claude apps (Anthropic recommended) \|
26	\| voyage-3 \| 1024 \| 32000 \| Claude apps, cost-effective \|
27	\| voyage-code-3 \| 1024 \| 32000 \| Code search \|
28	\| voyage-finance-2 \| 1024 \| 32000 \| Financial documents \|
29	\| voyage-law-2 \| 1024 \| 32000 \| Legal documents \|
30	\| text-embedding-3-large \| 3072 \| 8191 \| OpenAI apps, high accuracy \|
31	\| text-embedding-3-small \| 1536 \| 8191 \| OpenAI apps, cost-effective \|
32	\| bge-large-en-v1.5 \| 1024 \| 512 \| Open source, local deployment \|
33	\| all-MiniLM-L6-v2 \| 384 \| 256 \| Fast, lightweight \|
34	\| multilingual-e5-large \| 1024 \| 512 \| Multi-language \|
35
36	### 2. Embedding Pipeline
37
38	```
39	Document → Chunking → Preprocessing → Embedding Model → Vector
40	↓
41	[Overlap, Size] [Clean, Normalize] [API/Local]
42	```
43
44	## Templates and detailed worked examples
45
46	Full template library and detailed worked examples live in `references/details.md`. Read that file when you need the concrete templates.
47
48	## Best Practices
49
50	### Do's
51
52	- Match model to use case: Code vs prose vs multilingual
53	- Chunk thoughtfully: Preserve semantic boundaries
54	- Normalize embeddings: For cosine similarity search
55	- Batch requests: More efficient than one-by-one
56	- Cache embeddings: Avoid recomputing for static content
57	- Use Voyage AI for Claude apps: Recommended by Anthropic
58
59	### Don'ts
60
61	- Don't ignore token limits: Truncation loses information
62	- Don't mix embedding models: Incompatible vector spaces
63	- Don't skip preprocessing: Garbage in, garbage out
64	- Don't over-chunk: Lose important context
65	- Don't forget metadata: Essential for filtering and debugging
66