Large PDF library (1000+ docs) on a DGX Spark
Draft, untested Assembled from NVIDIA playbooks and owner reports. RAGShelf has not tested it end to end on a DGX Spark.
- Parse with Docling using the ghcr.io/docling-project/docling-serve-cu130 image, the build confirmed on GB10 (source). The cu126/cu128 images do not work on sm_121.
- Serve the LLM with vLLM from NVIDIA's playbook: gpt-oss-120b on one unit, or Qwen3-235B-A22B on a dual Spark.
- Serve bge-m3 embeddings and bge-reranker-v2-m3 locally (an owner-tested combination; source).
- Store chunks in Qdrant or Weaviate, with file name, path, and date metadata on each chunk.
- Front-end: Dify knowledge base or RAGFlow (community ARM64 build; source). Ingest in batches and watch memory, because the LLM, parser, and embedder share 128GB.
A one-command version of this stack: AGMind (community project, not reviewed by RAGShelf).
Last updated 2026-10-24
The RAGShelf weekly
New local RAG tools, releases, and tested setups. One email a week, no spam.