AutoGineer
Generated banner for Similarity Search Patterns
Generated logo for Similarity Search Patterns

Similarity Search Patterns

Seth HobsonCommunityCreated by Seth Hobson

  • Skill
  • Guidelines
  • Data and AI
  • MIT
  • No issues found
Download ZIP

From the author

Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance.
— the description by Seth Hobson

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to implementing similarity search, covering essential distance metrics, index types, and architectural patterns. It includes practical templates for integrating with vector database providers, offering guidance on how to select appropriate indexing strategies, manage search parameters, and implement hybrid search techniques to balance recall and latency.

What it does

  • Explains core distance metrics like Cosine, Euclidean, Dot Product, and Manhattan.
  • Compares index types including Flat, HNSW, and IVF+PQ based on search characteristics.
  • Provides code templates for vector database operations such as upsert, search, and deletion.
  • Demonstrates how to implement search with reranking using cross-encoders.
  • Outlines best practices for parameter tuning and filtering search spaces.

When to use it

  • When building semantic search or recommendation engines.
  • When implementing retrieval steps for RAG systems.
  • When choosing between different vector indexing strategies.
  • When integrating vector database operations into an application.

Advantages

  • Provides clear comparisons of indexing methods for different data scales.
  • Includes reusable code templates for common vector database workflows.
  • Offers actionable guidance on balancing search quality and system resources.
  • Covers both foundational concepts and practical implementation patterns.

Usage examples

“How do I choose the right index type for my vector database?”
The skill provides a comparison table showing that Flat indexes are best for small data, HNSW for medium-to-large, and IVF+PQ for very large datasets.
“Show me how to implement a search with reranking.”
The skill provides a code template demonstrating how to over-fetch results from a vector index and then refine them using a cross-encoder model.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/wshobson/similarity-search-patterns/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip similarity-search-patterns.zip -d .agents/skills

Documentation by Seth Hobson

Similarity Search Patterns

Patterns for implementing efficient similarity search in production systems.

When to Use This Skill

  • Building semantic search systems

  • Implementing RAG retrieval

  • Creating recommendation engines

  • Optimizing search latency

  • Scaling to millions of vectors

  • Combining semantic and keyword search

Core Concepts

1. Distance Metrics

| Metric | Formula | Best For | | ------------------ | ------------------ | --------------------- | --- | -------------- | | Cosine | 1 - (A·B)/(‖A‖‖B‖) | Normalized embeddings | | Euclidean (L2) | √Σ(a-b)² | Raw embeddings | | Dot Product | A·B | Magnitude matters | | Manhattan (L1) | Σ | a-b | | Sparse vectors |

2. Index Types
┌─────────────────────────────────────────────────┐
│                 Index Types                      │
├─────────────┬───────────────┬───────────────────┤
│    Flat     │     HNSW      │    IVF+PQ         │
│ (Exact)     │ (Graph-based) │ (Quantized)       │
├─────────────┼───────────────┼───────────────────┤
│ O(n) search │ O(log n)      │ O(√n)             │
│ 100% recall │ ~95-99%       │ ~90-95%           │
│ Small data  │ Medium-Large  │ Very Large        │
└─────────────┴───────────────┴───────────────────┘

Templates and detailed worked examples

Full template library and detailed worked examples live in references/details.md. Read that file when you need the concrete templates.

Best Practices

Do's

  • Use appropriate index - HNSW for most cases

  • Tune parameters - ef_search, nprobe for recall/speed

  • Implement hybrid search - Combine with keyword search

  • Monitor recall - Measure search quality

  • Pre-filter when possible - Reduce search space

Don'ts

  • Don't skip evaluation - Measure before optimizing

  • Don't over-index - Start with flat, scale up

  • Don't ignore latency - P99 matters for UX

  • Don't forget costs - Vector storage adds up

Files

LICENSE

MIT License

Copyright (c) 2024 Seth Hobson

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
Seth Hobson
Publisher
Seth Hobson (Community)
Source
wshobson/agents · plugins/llm-application-dev/skills/similarity-search-patterns · commit 4bde316
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Seth Hobson.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download similarity-search-patterns.zip