Qdrant Indexing Performance Optimization
GitHubOfficialFirst committed by anush008Distributed by GitHub
- Skill
- Tool integration
- Data and AI
- MIT
- No issues found
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a structured approach to troubleshooting slow data ingestion and indexing performance in Qdrant. It covers client-side batching strategies, server-side configuration adjustments for HNSW indexing, shard management, and optimization monitoring. It helps users identify whether performance issues stem from network overhead, hardware constraints, or suboptimal indexing parameters, while offering best practices for bulk uploads and multi-tenant collection management.
What it does
- Identifies bottlenecks in client-side batching and server-side ingestion.
- Provides configuration strategies to optimize HNSW build times and indexing thresholds.
- Guides users on monitoring optimizer status and segment merging progress.
- Offers solutions for multi-tenant collections to reduce global indexing overhead.
- Advises on hardware-specific optimizations like CPU thread allocation and disk I/O requirements.
- Highlights common pitfalls to avoid during bulk data operations.
When to use it
- When experiencing slow API upsert or upload speeds.
- When HNSW index construction times are excessively long.
- When the Qdrant optimizer appears stuck or reports errors.
- When planning large-scale bulk data migrations into a collection.
- When search performance is degraded following data ingestion.
Advantages
- Reduces indexing latency through targeted configuration tuning.
- Prevents common performance pitfalls like single-point uploads.
- Improves system stability by aligning indexing strategies with hardware capabilities.
- Provides actionable steps for both initial bulk loads and ongoing ingestion.
Usage examples
- “My Qdrant uploads are taking too long, what should I check?”
- The agent will guide you to check client-side batching (64-256 points per request), parallel upload streams, and server-side shard configuration.
- “How can I speed up HNSW index building for my collection?”
- The agent will suggest adjusting HNSW parameters like m and ef_construct, or using multi-tenant payload indexing to avoid global index builds.
Install
Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/github/qdrant-indexing-performance-optimization/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip qdrant-indexing-performance-optimization.zip -d .agents/skills
Documentation by anush008
What to Do When Qdrant Indexing Is Too Slow
Qdrant does NOT build HNSW indexes immediately. Small segments use brute-force until they exceed indexing_threshold_kb (default: 20 MB). Search during this window is slower by design, not a bug.
Understand the indexing optimizer Indexing optimizer
Uploads/Ingestion Too Slow
Use when: upload or upsert API calls are slow. Identify bottleneck: client-side (network, batching) vs server-side (CPU, disk I/O)
For client-side, optimize batching and parallelism:
Use batch upserts (64-256 points per request) Points API
Use 2-4 parallel upload streams
For server-side, optimize Qdrant configuration and indexing strategy:
Create more shards (3-12), each shard has an independent update worker Sharding
Create payload indexes before HNSW builds (needed for filterable vector index) Payload index
Suitable for initial bulk load of large datasets:
Disable HNSW during bulk load (set
indexing_threshold_kbvery high, restore after) Collection paramsSetting
m=0to disable HNSW is legacy, use highindexing_threshold_kbinstead
Careful, fast unindexed upload might temporarily use more RAM and degrade search performance until optimizer catches up.
See https://search.qdrant.tech/md/documentation/tutorials-develop/bulk-upload/
Optimizer Stuck or Taking Too Long
Use when: optimizer running for hours, not finishing.
Check actual progress via optimizations endpoint (v1.17+) Optimization monitoring
Large merges and HNSW rebuilds legitimately take hours on big datasets
Check CPU and disk I/O (HNSW is CPU-bound, merging is I/O-bound, HDD is not viable)
If
optimizer_statusshows an error, check logs for disk full or corrupted segments
HNSW Build Time Too High
Use when: HNSW index build dominates total indexing time.
Reduce
m(default 16, good for most cases, 32+ rarely needed) HNSW paramsReduce
ef_construct(100-200 sufficient) HNSW configKeep
max_indexing_threadsproportional to CPU cores ConfigurationUse GPU for indexing GPU indexing
HNSW index for multi-tenant collections
If you have a multi-tenant use case where all data is split by some payload field (e.g. tenant_id), you can avoid building a global HNSW index and instead rely on payload_m to build HNSW index only for subsets of data.
Skipping global HNSW index can significantly reduce indexing time.
See Multi-tenant collections for details.
Additional Payload Indexes Are Too Slow
Qdrant builds extra HNSW links for all payload indexes to ensure that quality of filtered vector search does not degrade.
Some payload indexes (e.g. text fields with long texts) can have a very high number of unique values per point, which can lead to long HNSW build time.
You can disable building extra HNSW links for specific payload index and instead rely on slightly slower query-time strategies like ACORN.
Read more about disabling extra HNSW links in documentation
Read more about ACORN in documentation
What NOT to Do
Do not create payload indexes AFTER HNSW is built (breaks filterable vector index)
Do not use
m=0for bulk uploads into an existing collection, it might drop the existing HNSW and cause long reindexingDo not upload one point at a time (per-request overhead dominates)
Files
- ATTRIBUTION.mdAdded by AutoGineer938 B
- LICENSELicence file added by AutoGineer1.0 KiB
- SKILL.md4.7 KiB
Requirements and compatibility
Written for: GitHub Copilot
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Security
No issues found by our static scan· No issues found
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- First committed by
anush008
- Publisher
- GitHub (Official)
- Distributed by
- GitHub
- Source
- github/awesome-copilot · skills/qdrant-performance-optimization/indexing-performance-optimization · commit 797337f
- Licence
- MIT License (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
- Contributors
- anush008
AutoGineer is not affiliated with or endorsed by anush008 or GitHub.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download qdrant-indexing-performance-optimization.zip