Local RAG vs cloud file search

Every big AI lab now sells "upload your files and ask questions". Here's how that compares with running it yourself. All prices come from each provider's official page, fetched 2026-10-27. Prices change often, so click through before you decide. Want your own numbers? Use the cost calculator.

At a glance

	Local (RAGShelf tools)	OpenAI File search	Google Gemini File Search	Anthropic Claude	xAI Grok Collections
Hosted retrieval?	You run it: AnythingLLM, Open WebUI, RAGFlow, more	Yes: vector stores + file_search tool	Yes: File Search stores	No hosted vector store on the pricing page. You bring retrieval; Anthropic publishes the Contextual Retrieval method	Yes: Collections + collections_search tool
Model price (per 1M tokens, in / out)	$0. You pay for hardware and power	gpt-6-luna $0.1 in / $0.5 out, gpt-6.1-terra $1 in / $5 out, gpt-6.1-sol $2 in / $10 out, gpt-6-astra $10 in / $50 out	gemini-3.5-flash-lite $0.3 in / $2.5 out, gemini-3.8-flash $0.75 in / $3.75 out, gemini-3.1-pro-preview $2 in / $12 out, gemini-4-argon $2 in / $10 out	claude-haiku-5.5 $0.1 in / $0.5 out, claude-sonnet-5.5 $2 in / $10 out, claude-opus-5.5 $4 in / $20 out	grok-4.3 $1.25 in / $2.5 out, grok-4.7 $2 in / $6 out
Retrieval fees	None	Storage $0.10/GB per day (1 GB free); $2.50 per 1k file search calls	Indexing embeddings $0.15/1M tokens; storage and query embeddings free; retrieved text billed as input	None from Anthropic (your own stack)	Collection storage $0.10/GiB per day; $2.50 per 1k Collections Search calls
Data used for training?	No. Nothing leaves your machine	API data not used for training unless you opt in	Paid tier: no. Free tier: yes ("used to improve our products")	Commercial API: not used for training without permission	Not used for training without explicit permission
Retention	Yours	Abuse-monitoring logs up to 30 days by default	Files in a File Search store kept until you delete them; raw File API uploads deleted after 48 hours	Prompts/outputs not retained by default (some models need 30 days); ZDR available	Requests stored 30 days for abuse auditing; ZDR available but turns off Collections
Limits	Your disk and RAM	See OpenAI docs	100 MB per file; total stores 1 GB free tier, 10 GB Tier 1, 100 GB Tier 2, 1 TB Tier 3 (store size is about 3x your input)	Context window of the model you pick	100 MB per file; credits required to upload
Setup effort	Desktop app in minutes (AnythingLLM, Jan) up to a weekend (Docker stacks)	API code, minutes	API code, minutes	Most work: you build retrieval	API code, minutes
Sources	RAGShelf directory	pricing · your data	pricing · File Search docs	pricing · data retention · training	pricing · Collections · security FAQ

Promo pricing: Price through Dec 31, 2026; $1.50 / $7.50 from Jan 1, 2027 (Gemini 3.8 Flash); Price through Nov 30, 2026; $4 / $20 from Dec 1, 2026 (Gemini 4 Argon). Claude Haiku 5.5 price is for prompts up to 100k tokens; Grok prices are for prompts under 200k tokens. Standard (not batch) rates shown. Checked 2026-10-27.

When local wins

Private or regulated files you can't upload.
Big libraries you query a lot: no per-question or per-GB-day fees.
You already own a decent GPU, Mac or DGX Spark.
Offline use.

When cloud wins

A few hundred questions a month: pennies, no hardware.
You need the strongest frontier model.
No time to run a server.

Hybrid: local retrieval + cloud model

Keep the library and the search index on your machine, and send only the question plus the few retrieved passages to a cloud model. You get frontier-model answers without uploading the whole archive.

Desktop app + API key

AnythingLLM, Open WebUI, Jan or Cherry Studio index locally and can call OpenAI, Gemini, Claude or Grok for the answer step.

Local MCP server → Claude/ChatGPT/Cursor

Run a tool with an MCP server (DocsGPT, RAGFlow, Onyx) and connect your AI app to it.

Copy Contextual Retrieval locally

Anthropic reports that adding a short context blurb to each chunk before embedding, plus BM25, cut failed retrievals by 49%, and by 67% with reranking (source). You can build this with LlamaIndex or Haystack and a local model from our models page.

Hybrid caveat: the retrieved passages do go to the cloud provider, under the data policies above.