ToolsRecipesCompareCategoriesHardwareDGX SparkModelsMCPLocal vs cloudCost calculatorPeopleQuestionsSubmitAboutSponsorNewsletterFollow on X

← All guides

Hardware › Guides

Local AI on a small box or under $2,500

Prices checked 2026-11-08

You do not need a DGX Spark or a multi-GPU tower to run local document Q&A. This page maps mini-PCs with shared APU memory, used discrete GPUs, and Mac minis that fit under about $2,500, then points at RAGShelf tools that can run on that class of hardware. Prices are retail or list prices checked on the dates below—not negotiated deals, and not a promise of stock.

What "small box" usually means

  • APU / mini-PC. CPU and GPU share one memory pool (for example 24 GB LPDDR5X). That pool has to hold the OS, the model weights in use, KV cache, and your RAG index process. A 24 GB machine is workable for small-to-mid models at low bit-width; it is tight for large context plus a heavy embedder at the same time.
  • Used discrete GPU. More VRAM headroom for one model, but you still need a host PC, power, and cooling. Mining / data-center cards can be cheap per GB and risky (unlocks, returns, failure rates).
  • Mac mini. Unified memory is shared the same way as an APU. Apple Silicon is a common path for local chat and RAG apps with Metal backends.

Example systems under $2,500 (checked 2026-11-08)

ClassExampleMemory / notesPrice checkedSource
Mini-PC APUBeelink SER9 Pro (Ryzen 7 H 255, Radeon 780M)24 GB LPDDR5X (not user-expandable on this class), 1 TB SSD configs common$769 (list / sale pair showed $769 / $919)Beelink US SE series, checked 2026-11-08
Mini-PC APUBeelink SER9 24 GB / 1 TB listingsSame memory class; third-party listings varyAbout $689–$699 on Amazon search results the same dayAmazon product listings (SER9 / SER9 Pro 24 GB), checked 2026-11-08 — confirm live price before buying
Mac miniMac mini with M6Unified memory; configure more RAM if you plan larger local modelsFrom $899Apple Store Mac mini, checked 2026-11-08
Used GPU pathNVIDIA CMP 170HX (unlocked community path)High HBM capacity after unlock; Linux-oriented, PCIe limits, riskSee live CMP page (median eBay asks were in the mid-$3k range earlier in Nov — often above this budget alone)RAGShelf CMP 170HX guide

Shipping, tax and stock change quickly. Re-check the linked store pages. A CMP 170HX card alone often exceeds $2,500 in recent eBay medians tracked on our hardware page, so it is listed as a related option, not a fit for this budget band.

What model sizes tend to fit

  • ~8–16 GB usable for weights on a 24 GB shared-memory box after OS and apps: think small instruct models and low-bit quantizations, not full 70B at high precision.
  • Mac unified memory scales with the config you buy; leave headroom for the RAG app and embeddings, not just the chat model.
  • Used mid-range GPUs (when priced under the remaining budget) are for people who already have a desktop chassis and PSU.

For model picks aimed at document Q&A, start at /models/. For monthly cost tradeoffs versus APIs, use the cost calculator.

Which RAG tools make sense here

Prefer tools tagged as runnable on CPU or modest GPU, with a fully offline path once local models are configured. Good starting points on RAGShelf:

Always check each tool page for hardware notes and whether defaults call a cloud embedder.

Related on RAGShelf

Hardware hub · CMP 170HX · Cost calculator · DGX Spark (above this budget) · Why retrieval fails

Sources

  • Beelink US SE series — SER9 Pro H 255 price $769 / $919 (checked 2026-11-08)
  • Apple Buy Mac mini — from $899 (checked 2026-11-08)
  • Amazon SER9 / SER9 Pro 24 GB listings — about $689–$699 in search results the same day (confirm at checkout)
  • RAGShelf CMP 170HX for used high-memory GPU caveats

The RAGShelf weekly

New local RAG tools, releases, and tested setups. One email a week, no spam.

We use your email only for the newsletter. Privacy policy

Updates on X: @mkjerman