All tools
RAGShelf compares 38 self-hosted and desktop tools that let a local model answer questions from your PDFs, Office files, scans, and notes, without sending them to the cloud. Filter by license, hardware, document type, and whether it can run fully offline.
Browse setup recipes Submit a tool
📬 Get one new local RAG tool every week. Free. Unsubscribe anytime.
Updates on X: @mkjerman
🗂 Browse by category: PDFs · Ollama · Desktop apps (beginner) · Docker servers · Knowledge graph · Scans & OCR · Team KB
➕ Built a local RAG tool? Submit it to RAGShelf. Free listings, reviewed weekly.
🟩 New: Local RAG on NVIDIA DGX Spark: which tools work on GB10/ARM64, what fits in 128GB, 256GB and 512GB, and single vs dual vs quad Spark stacks.
🟨 New: Best local models (Oct 2026): the top 10 open models for document Q&A, with memory needs, hardware fit and sources.
🟦 New: People to follow on X: 37 active accounts covering local models, inference tooling, RAG, research and the DGX Spark community.
Filters only use what each listing's data supports; tools with unknown hardware or use case stay untagged. Difficulty: desktop app = Easy, server/Docker = Medium, developer library = Hard. Filtered views are shareable: copy the URL.
How we know: Docs only = taken from the project's own README/docs, not tested by us. Community report = an owner published a working (or problem) report; links to the source. Vendor guide = an official NVIDIA playbook exists. We have not hands-on tested any tool yet, so nothing here says "Tested by RAGShelf".
Most clicked this week
Outbound clicks (GitHub, website, install copy) over the last 7 days. Counts only, no personal data.
| Tool | Hardware | Fully offline | Retrieval | ★ Stars | Last checked |
|---|---|---|---|---|---|
| AirLLM New Active Difficulty: Hard Docs only | Python inference library for low-VRAM GPUs (project reports 70B in ~4GB); CPU inference supported; macOS on Apple silicon (Llama-style models only); needs lots of fast disk | ✅ Yes | No retrieval (runner/parser) | 40,814 | 2026-10-26 |
| AnythingLLM Active Difficulty: Easy DGX Spark: Reported working Community report | Desktop or Docker; built-in engine runs CPU or GPU (NVIDIA CUDA, AMD/Intel Vulkan) | ✅ Yes | Vector search | 67,579 | 2026-10-26 |
| Cherry Studio Active Difficulty: Easy Docs only | Desktop app (Windows, macOS, Linux) | ✅ Yes | Hybrid (vector + keyword) | 52,997 | 2026-10-26 |
| Dify Active Difficulty: Medium DGX Spark: Reported working Community report | Docker; minimum 2 CPU cores, 4GB RAM | ✅ Yes | Hybrid (vector + keyword) | 159,592 | 2026-10-26 |
| Docling Active DGX Spark: Reported working Community report | CPU OK, GPU faster | ✅ Yes | No retrieval (runner/parser) | 69,615 | 2026-10-26 |
| DocsGPT New Active Difficulty: Medium Docs only | Docker (installer runs Docker Compose; pip, Kubernetes and air-gapped installs documented) | ✅ Yes | Vector search Graph RAG Agentic / iterative | 18,335 | 2026-10-26 |
| Flowise Active Difficulty: Medium Docs only | Node server | ✅ Yes | Vector search | 55,529 | 2026-10-26 |
| GPT4All (LocalDocs) Stale (no release in 12+ months) Difficulty: Easy Docs only | Desktop; CPU OK | ✅ Yes | Vector search | 77,356 | 2026-10-26 |
| h2oGPT Stale (no release in 12+ months) Docs only | GPU recommended | ✅ Yes | Vector search | 11,932 | 2026-10-26 |
| Haystack Active Difficulty: Hard DGX Spark: Reported working Community report | Framework | ✅ Yes | Vector search Agentic / iterative | 26,847 | 2026-10-26 |
| Jan Active Difficulty: Easy Docs only | Desktop; CPU/GPU (Vulkan, Metal, CUDA, ROCm). macOS: 8GB RAM for 3B, 16GB for 7B, 32GB for 13B models | ✅ Yes | Vector search Full-context Agentic / iterative | 45,092 | 2026-10-26 |
| Khoj Active Difficulty: Medium Docs only | Server; GPU optional | ✅ Yes | Vector search | 37,834 | 2026-10-26 |
| KoboldCpp Active Difficulty: Easy Docs only | Single binary; CPU/GPU | ✅ Yes | Vector search | 12,065 | 2026-10-26 |
| Kotaemon Active Difficulty: Medium Docs only | Docker; GPU optional | ✅ Yes | Hybrid (vector + keyword) Agentic / iterative Graph RAG | 25,829 | 2026-10-26 |
| Langflow Active Difficulty: Medium Docs only | Python server | ✅ Yes | Vector search | 156,030 | 2026-10-26 |
| LibreChat No tracked release Difficulty: Medium DGX Spark: Reported working Community report | Server + separate RAG API with pgvector | ⚠️ Needs config | Vector search Agentic / iterative Full-context | 46,342 | 2026-10-26 |
| LightRAG Active Docs only | Depends on chosen LLM; graph extraction wants a 14B+ model | ✅ Yes | Graph RAG Vector search | 40,344 | 2026-10-26 |
| LlamaIndex Active Difficulty: Hard Docs only | Framework | ✅ Yes | Vector search Agentic / iterative | 52,670 | 2026-10-26 |
| LocalAI Active DGX Spark: Reported working Community report | CPU/GPU | ✅ Yes | No retrieval (runner/parser) | 49,970 | 2026-10-26 |
| LocalGPT No tracked release Docs only | GPU recommended | ✅ Yes | Hybrid (vector + keyword) | 22,189 | 2026-10-26 |
| MaxKB New Active Difficulty: Medium Docs only | Docker (PostgreSQL + pgvector) | ✅ Yes | Agentic / iterative | 23,039 | 2026-10-26 |
| Microsoft GraphRAG Active Docs only | Indexing is LLM-heavy | ✅ Yes | Graph RAG | 36,452 | 2026-10-26 |
| Msty Studio No tracked release Difficulty: Easy Docs only | Desktop app | ⚠️ Needs config | Hybrid (vector + keyword) Full-context Agentic / iterative | — | 2026-10-26 |
| Ollama Active DGX Spark: Official NVIDIA playbook Vendor guide | CPU/GPU/Apple | ✅ Yes | No retrieval (runner/parser) | 184,845 | 2026-10-26 |
| Onyx Active Difficulty: Medium Docs only | Docker; 16GB RAM suggested | ✅ Yes | Hybrid (vector + keyword) Agentic / iterative | 32,996 | 2026-10-26 |
| Open Notebook Active Difficulty: Medium DGX Spark: Reported working Community report | Docker | ✅ Yes | Hybrid (vector + keyword) | 40,601 | 2026-10-26 |
| Open WebUI Active Difficulty: Medium DGX Spark: Official NVIDIA playbook Vendor guide | Light server; LLM via Ollama | ✅ Yes | Hybrid (vector + keyword) Full-context | 156,807 | 2026-10-26 |
| paperless-gpt Active Docs only | Needs Paperless-ngx + LLM | ✅ Yes | No retrieval (runner/parser) | 2,789 | 2026-10-26 |
| Paperless-ngx Active Docs only | Low (Raspberry Pi OK) | ✅ Yes | No retrieval (runner/parser) | 47,106 | 2026-10-26 |
| PipesHub Active Difficulty: Medium Docs only | Docker stack | ✅ Yes | Graph RAG Agentic / iterative | 3,850 | 2026-10-26 |
| PrivateGPT Active Docs only | 8GB+ RAM; GPU recommended | ✅ Yes | Vector search Agentic / iterative | 57,615 | 2026-10-26 |
| Quivr Stale (no release in 12+ months) Difficulty: Hard Docs only | Python library | ⚠️ Needs config | Hybrid (vector + keyword) | 39,604 | 2026-10-26 |
| R2R Stale (no release in 12+ months) Difficulty: Medium Docs only | Docker, Postgres | ⚠️ Needs config | Hybrid (vector + keyword) Graph RAG Agentic / iterative | 8,043 | 2026-10-26 |
| RAG Web UI Active Difficulty: Medium Docs only | Docker | ✅ Yes | Vector search | 3,301 | 2026-10-26 |
| RAGFlow Active Difficulty: Medium DGX Spark: Known issues Community report | 4 cores, 16GB RAM, 50GB disk recommended (Docker) | ✅ Yes | Hybrid (vector + keyword) Graph RAG Agentic / iterative Layout / ToC tree | 92,895 | 2026-10-26 |
| SurfSense New Active Difficulty: Easy Docs only | Desktop app (Windows x64, macOS Apple Silicon, Linux x64); pick a local or remote model | ✅ Yes | Agentic / iterative | 16,605 | 2026-10-26 |
| Unstructured (OSS) Active Docs only | CPU; hi_res needs more | ✅ Yes | No retrieval (runner/parser) | 15,626 | 2026-10-26 |
| Verba Stale (no release in 12+ months) Difficulty: Medium Docs only | Docker + Weaviate | ✅ Yes | Vector search | 7,693 | 2026-10-26 |
Missing a tool? Submit a tool Free listing, reviewed weekly.
"Fully offline: yes" means it can run with no cloud calls once configured with local models. "Needs config" means a default (e.g. OpenAI embeddings or a cloud reranker) must be swapped out. New = added to RAGShelf in the last 14 days. Last checked = when we last reviewed the listing. Stale = no release in over 12 months. Stars and last commit (last push) come from the GitHub API, fetched '+GHDATE+'; click a column header to sort.
The RAGShelf weekly
New local RAG tools, releases, and tested setups. One email a week, no spam.
Updates on X: @mkjerman