Local RAG tools for PDFs
Tools tagged for chatting with documents whose listing says they read PDFs. 16 tools, sorted by GitHub stars.
📬 Get the weekly list of new local RAG tools and releases. Free, one email a week.
| Tool | Hardware | Fully offline |
|---|---|---|
| Open WebUI Active Difficulty: Medium Vendor guide | Light server; LLM via Ollama | ✅ Yes |
| RAGFlow Active Difficulty: Medium Community report | 4 cores, 16GB RAM, 50GB disk recommended (Docker) | ✅ Yes |
| GPT4All (LocalDocs) Stale (no release in 12+ months) Difficulty: Easy Docs only | Desktop; CPU OK | ✅ Yes |
| AnythingLLM Active Difficulty: Easy Community report | Desktop or Docker; built-in engine runs CPU or GPU (NVIDIA CUDA, AMD/Intel Vulkan) | ✅ Yes |
| PrivateGPT Active Docs only | 8GB+ RAM; GPU recommended | ✅ Yes |
| Cherry Studio Active Difficulty: Easy Docs only | Desktop app (Windows, macOS, Linux) | ✅ Yes |
| LibreChat No tracked release Difficulty: Medium Community report | Server + separate RAG API with pgvector | ⚠️ Needs config |
| Khoj Active Difficulty: Medium Docs only | Server; GPU optional | ✅ Yes |
| Kotaemon Active Difficulty: Medium Docs only | Docker; GPU optional | ✅ Yes |
| LocalGPT No tracked release Docs only | GPU recommended | ✅ Yes |
| DocsGPT Active Difficulty: Medium Docs only | Docker (installer runs Docker Compose; pip, Kubernetes and air-gapped installs documented) | ✅ Yes |
| SurfSense Active Difficulty: Easy Docs only | Desktop app (Windows x64, macOS Apple Silicon, Linux x64); pick a local or remote model | ✅ Yes |
| h2oGPT Stale (no release in 12+ months) Docs only | GPU recommended | ✅ Yes |
| Verba Stale (no release in 12+ months) Difficulty: Medium Docs only | Docker + Weaviate | ✅ Yes |
| RAG Web UI Active Difficulty: Medium Docs only | Docker | ✅ Yes |
| Msty Studio No tracked release Difficulty: Easy Docs only | Desktop app | ⚠️ Needs config |
How we know: Docs only = taken from the project's own README/docs, not tested by us. Community report = an owner published a working (or problem) report; links to the source. Vendor guide = an official NVIDIA playbook exists. We have not hands-on tested any tool yet, so nothing here says "Tested by RAGShelf".
Setup recipes
- Chat with my docs on a laptop no gpu
- 300 long pdfs with citations on a 24gb card
- Fully private health legal records
- Already run open webui
- Apple silicon mac
- Private document qa on a dgx spark
- Large pdf library 1000 docs on a dgx spark
Comparisons
- AnythingLLM vs Open WebUI
- AnythingLLM vs RAGFlow
- AnythingLLM vs PrivateGPT
- RAGFlow vs Dify
- Open WebUI vs LibreChat
- Jan vs GPT4All (LocalDocs)
- Khoj vs Open Notebook
Groupings use only what each listing's data supports. Last updated 2026-10-26.
The RAGShelf weekly
New local RAG tools, releases, and tested setups. One email a week, no spam.