Generated banner for Qdrant Indexing Performance Optimization
Generated logo for Qdrant Indexing Performance Optimization

Qdrant Indexing Performance Optimization

GitHubOfficialFirst committed by anush008Distributed by GitHub

  • Skill
  • Tool integration
  • Data and AI
  • MIT
  • No issues found

From the author

Diagnoses and fixes slow Qdrant indexing and data ingestion. Use when someone reports 'uploads are slow', 'indexing takes forever', 'optimizer is stuck', 'HNSW build time too long', or 'data uploaded but search is bad'. Also use when optimizer status shows errors, segments won't merge, or indexing threshold questions arise.
— the description by anush008

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to troubleshooting slow data ingestion and indexing performance in Qdrant. It covers client-side batching strategies, server-side configuration adjustments for HNSW indexing, shard management, and optimization monitoring. It helps users identify whether performance issues stem from network overhead, hardware constraints, or suboptimal indexing parameters, while offering best practices for bulk uploads and multi-tenant collection management.

What it does

  • Identifies bottlenecks in client-side batching and server-side ingestion.
  • Provides configuration strategies to optimize HNSW build times and indexing thresholds.
  • Guides users on monitoring optimizer status and segment merging progress.
  • Offers solutions for multi-tenant collections to reduce global indexing overhead.
  • Advises on hardware-specific optimizations like CPU thread allocation and disk I/O requirements.
  • Highlights common pitfalls to avoid during bulk data operations.

When to use it

  • When experiencing slow API upsert or upload speeds.
  • When HNSW index construction times are excessively long.
  • When the Qdrant optimizer appears stuck or reports errors.
  • When planning large-scale bulk data migrations into a collection.
  • When search performance is degraded following data ingestion.

Advantages

  • Reduces indexing latency through targeted configuration tuning.
  • Prevents common performance pitfalls like single-point uploads.
  • Improves system stability by aligning indexing strategies with hardware capabilities.
  • Provides actionable steps for both initial bulk loads and ongoing ingestion.

Usage examples

“My Qdrant uploads are taking too long, what should I check?”
The agent will guide you to check client-side batching (64-256 points per request), parallel upload streams, and server-side shard configuration.
“How can I speed up HNSW index building for my collection?”
The agent will suggest adjusting HNSW parameters like m and ef_construct, or using multi-tenant payload indexing to avoid global index builds.

Install

  1. Download the skill as a ZIP:

    https://mp-staging.autogineer.com/agent-skills/github/qdrant-indexing-performance-optimization/download
  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip qdrant-indexing-performance-optimization.zip -d .agents/skills

Documentation by anush008

What to Do When Qdrant Indexing Is Too Slow

Qdrant does NOT build HNSW indexes immediately. Small segments use brute-force until they exceed indexing_threshold_kb (default: 20 MB). Search during this window is slower by design, not a bug.

Uploads/Ingestion Too Slow

Use when: upload or upsert API calls are slow. Identify bottleneck: client-side (network, batching) vs server-side (CPU, disk I/O)

For client-side, optimize batching and parallelism:

  • Use batch upserts (64-256 points per request) Points API

  • Use 2-4 parallel upload streams

For server-side, optimize Qdrant configuration and indexing strategy:

  • Create more shards (3-12), each shard has an independent update worker Sharding

  • Create payload indexes before HNSW builds (needed for filterable vector index) Payload index

Suitable for initial bulk load of large datasets:

  • Disable HNSW during bulk load (set indexing_threshold_kb very high, restore after) Collection params

  • Setting m=0 to disable HNSW is legacy, use high indexing_threshold_kb instead

Careful, fast unindexed upload might temporarily use more RAM and degrade search performance until optimizer catches up.

See https://search.qdrant.tech/md/documentation/tutorials-develop/bulk-upload/

Optimizer Stuck or Taking Too Long

Use when: optimizer running for hours, not finishing.

  • Check actual progress via optimizations endpoint (v1.17+) Optimization monitoring

  • Large merges and HNSW rebuilds legitimately take hours on big datasets

  • Check CPU and disk I/O (HNSW is CPU-bound, merging is I/O-bound, HDD is not viable)

  • If optimizer_status shows an error, check logs for disk full or corrupted segments

HNSW Build Time Too High

Use when: HNSW index build dominates total indexing time.

HNSW index for multi-tenant collections

If you have a multi-tenant use case where all data is split by some payload field (e.g. tenant_id), you can avoid building a global HNSW index and instead rely on payload_m to build HNSW index only for subsets of data. Skipping global HNSW index can significantly reduce indexing time.

See Multi-tenant collections for details.

Additional Payload Indexes Are Too Slow

Qdrant builds extra HNSW links for all payload indexes to ensure that quality of filtered vector search does not degrade. Some payload indexes (e.g. text fields with long texts) can have a very high number of unique values per point, which can lead to long HNSW build time.

You can disable building extra HNSW links for specific payload index and instead rely on slightly slower query-time strategies like ACORN.

Read more about disabling extra HNSW links in documentation

Read more about ACORN in documentation

What NOT to Do

  • Do not create payload indexes AFTER HNSW is built (breaks filterable vector index)

  • Do not use m=0 for bulk uploads into an existing collection, it might drop the existing HNSW and cause long reindexing

  • Do not upload one point at a time (per-request overhead dominates)

Files

Requirements and compatibility

Written for: GitHub Copilot

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

First committed by
GitHub avatar of anush008anush008
Publisher
GitHub (Official)
Distributed by
GitHub
Source
github/awesome-copilot · skills/qdrant-performance-optimization/indexing-performance-optimization · commit 797337f
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced
Contributors
anush008

AutoGineer is not affiliated with or endorsed by anush008 or GitHub.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download qdrant-indexing-performance-optimization.zip