All tools
RAGShelf compares 34 self-hosted and desktop tools that let a local model answer questions from your PDFs, Office files, scans, and notes, without sending them to the cloud. Filter by license, hardware, document type, and whether it can run fully offline.
Browse setup recipes Submit a tool
➕ Built a local RAG tool? Submit it to RAGShelf. Free listings, reviewed weekly.
🟩 New: Local RAG on NVIDIA DGX Spark: which tools work on GB10/ARM64, what fits in 128GB, 256GB and 512GB, and single vs dual vs quad Spark stacks.
🟨 New: Best local models (Oct 2026): the top 10 open models for document Q&A, with memory needs, hardware fit and sources.
🟦 New: People to follow on X: 34 active accounts covering local models, inference tooling, RAG, research and the DGX Spark community.
Filters only use what each listing's data supports; tools with unknown hardware or use case stay untagged. Difficulty: desktop app = Easy, server/Docker = Medium, developer library = Hard. Filtered views are shareable: copy the URL.
| Tool | Hardware | Fully offline | ★ Stars |
|---|---|---|---|
| AnythingLLM Active Difficulty: Easy DGX Spark: Reported working | Desktop or Docker; built-in engine runs CPU or GPU (NVIDIA CUDA, AMD/Intel Vulkan) | ✅ Yes | 67,552 |
| Cherry Studio Active Difficulty: Easy | Desktop app (Windows, macOS, Linux) | ✅ Yes | 52,973 |
| Dify Active Difficulty: Medium DGX Spark: Reported working | Docker; minimum 2 CPU cores, 4GB RAM | ✅ Yes | 159,523 |
| Docling Active DGX Spark: Reported working | CPU OK, GPU faster | ✅ Yes | 69,582 |
| Flowise Active Difficulty: Medium | Node server | ✅ Yes | 55,528 |
| GPT4All (LocalDocs) Stale (no release in 12+ months) Difficulty: Easy | Desktop; CPU OK | ✅ Yes | 77,357 |
| h2oGPT Stale (no release in 12+ months) | GPU recommended | ✅ Yes | 11,933 |
| Haystack Active Difficulty: Hard DGX Spark: Reported working | Framework | ✅ Yes | 26,842 |
| Jan Active Difficulty: Easy | Desktop; CPU/GPU (Vulkan, Metal, CUDA, ROCm). macOS: 8GB RAM for 3B, 16GB for 7B, 32GB for 13B models | ✅ Yes | 45,084 |
| Khoj Active Difficulty: Medium | Server; GPU optional | ✅ Yes | 37,827 |
| KoboldCpp Active Difficulty: Easy | Single binary; CPU/GPU | ✅ Yes | 12,064 |
| Kotaemon Active Difficulty: Medium | Docker; GPU optional | ✅ Yes | 25,829 |
| Langflow Active Difficulty: Medium | Python server | ✅ Yes | 156,018 |
| LibreChat No tracked release Difficulty: Medium DGX Spark: Reported working | Server + separate RAG API with pgvector | ⚠️ Needs config | 46,317 |
| LightRAG Active | Depends on chosen LLM; graph extraction wants a 14B+ model | ✅ Yes | 40,325 |
| LlamaIndex Active Difficulty: Hard | Framework | ✅ Yes | 52,663 |
| LocalAI Active DGX Spark: Reported working | CPU/GPU | ✅ Yes | 49,955 |
| LocalGPT No tracked release | GPU recommended | ✅ Yes | 22,189 |
| Microsoft GraphRAG Active | Indexing is LLM-heavy | ✅ Yes | 36,445 |
| Msty Studio No tracked release Difficulty: Easy | Desktop app | ⚠️ Needs config | — |
| Ollama Active DGX Spark: Official NVIDIA playbook | CPU/GPU/Apple | ✅ Yes | 184,750 |
| Onyx Active Difficulty: Medium | Docker; 16GB RAM suggested | ✅ Yes | 32,982 |
| Open Notebook Active Difficulty: Medium DGX Spark: Reported working | Docker | ✅ Yes | 40,578 |
| Open WebUI Active Difficulty: Medium DGX Spark: Official NVIDIA playbook | Light server; LLM via Ollama | ✅ Yes | 156,748 |
| paperless-gpt Active | Needs Paperless-ngx + LLM | ✅ Yes | 2,788 |
| Paperless-ngx Active | Low (Raspberry Pi OK) | ✅ Yes | 47,094 |
| PipesHub Active Difficulty: Medium | Docker stack | ✅ Yes | 3,848 |
| PrivateGPT Active | 8GB+ RAM; GPU recommended | ✅ Yes | 57,613 |
| Quivr Stale (no release in 12+ months) Difficulty: Hard | Python library | ⚠️ Needs config | 39,603 |
| R2R Stale (no release in 12+ months) Difficulty: Medium | Docker, Postgres | ⚠️ Needs config | 8,043 |
| RAG Web UI Active Difficulty: Medium | Docker | ✅ Yes | 3,301 |
| RAGFlow Active Difficulty: Medium DGX Spark: Known issues | 4 cores, 16GB RAM, 50GB disk recommended (Docker) | ✅ Yes | 92,851 |
| Unstructured (OSS) Active | CPU; hi_res needs more | ✅ Yes | 15,623 |
| Verba Stale (no release in 12+ months) Difficulty: Medium | Docker + Weaviate | ✅ Yes | 7,693 |
Missing a tool? Submit a tool Free listing, reviewed weekly.
"Fully offline: yes" means it can run with no cloud calls once configured with local models. "Needs config" means a default (e.g. OpenAI embeddings or a cloud reranker) must be swapped out. Stale = no release in over 12 months. Stars and last commit (last push) come from the GitHub API, fetched '+GHDATE+'; click a column header to sort.
The RAGShelf weekly
New local RAG tools, releases, and tested setups. One email a week, no spam.