Recommendation for RAG / Long context
RAG & Long Context
The best LLM for RAG & long context is MoonshotAI's Kimi K3, thanks to its massive 1 million token context window that genuinely supports heavy document workloads. For builders prioritizing quality with large contexts, Claude Opus 4.6 and Claude Fable 5 rank as the top overall models on LMArena, with Fable 5 demonstrating strong performance even beyond 250k tokens on creative workloads. DeepSeek V4 Flash earns a spot as the budget-conscious choice, offering acceptable RAG performance with low hallucination rates at a fraction of the cost. Claude Opus 4.7 and GPT-5.4 serve as solid frontier options for fact-checking pipelines, while Gemma 4 31B is the pick for self-hosted deployments processing 150k+ tokens. The right choice depends heavily on whether you need maximum context length, top-tier reasoning quality, or cost-efficient inference at scale.
About this recommendation
- Updated
- Jul 17, 2026
- Evidence through
- Jul 17, 2026
- Sources
- 14
- Revision
- v1
Kimi K3 is purpose-built for extreme long-context workloads that would break other models. Its 1M token window ranks it as the top choice for RAG systems processing million-token documents. Pricing at $3-$15 per 1M tokens is competitive with Anthropic's Sonnet series given the context capacity.
Best when: You need to process massive documents exceeding 500k tokens in a single context window.
Tips
- 1M token context window supports massive workloads like EDA design and large codebases.
- Pricing at $3-$15 per 1M tokens matches Anthropic's Sonnet series.
- Ranks #8 of 49 on LMArena with Elo 1473, showing competitive overall quality.
Watch out for
- Pricing is extremely high for a Chinese open-weight model if you compare strictly on cost rather than capability.
Claude Opus 4.6 holds the #1 spot on LMArena, making it the top-ranked model for general quality. While the evidence here focuses on rankings rather than explicit context tests, its position above Fable 5 suggests it handles complex reasoning tasks as well as any frontier model available.
Best when: You need the highest overall model quality and are willing to pay frontier prices.
Tips
- Ranks #1 of 49 on LMArena with Elo 1501, the highest score observed.
Claude Fable 5 ranks #2 on LMArena and has been tested extensively on poetry collections exceeding 250k tokens. It demonstrates strong capability to analyze and synthesize meaning from large creative works, making it a strong choice for qualitative long-context tasks.
Best when: Your long-context work involves qualitative analysis, creative content, or synthesis over large document collections.
Tips
- Ranks #2 of 49 on LMArena with Elo 1493.
- Successfully processes and analyzes poetry collections spanning over 250k tokens.
- Handles complex qualitative tasks like evaluating author identity from unedited collections.
Watch out for
- Long-context testing was on creative writing rather than technical or code-heavy documents.
DeepSeek V4 Flash is the practical choice for cost-conscious RAG deployments. Users report it finds the right information with few hallucinations, making it acceptable for production pipelines where inference cost matters more than peak quality.
Best when: You need RAG at scale and want the cheapest model that still delivers acceptable accuracy.
Tips
- Cheapest model tested that still returns acceptable replies with few hallucinations.
- Successfully used for agents, sub-agents, RAG, and dynamic intent detection systems.
Watch out for
- Processing a 64k agentic context results in 2+ minutes waiting for the first token.
Claude Opus 4.7 ranks #3 on LMArena and appears in fact-checking benchmarks alongside GPT-5.4. It serves as a strong frontier parametric model, though the evidence shows that even top models disagree frequently on factual claims.
Best when: You need a high-quality parametric model for fact-checking or retrieval pipelines.
Tips
- Ranks #3 of 49 on LMArena with Elo 1490.
- Included in a five-model frontier comparison for fact-checking tasks.
Watch out for
- Part of a panel where 67% of real user claims had at least one model disagree with the majority.
Gemma 4 31B is the strongest self-hosted option for long-context workloads. Full precision with full context handles 150k+ business documents reasonably well, but MoE variants or quantized versions show clear logical mistakes and omissions on real tasks.
Best when: You must self-host and need to process 150k+ context with your own hardware.
Tips
- Full precision model handles long business contexts of 150k+ tokens reasonably well.
- Runs at 30+ tokens per second sustained on consumer hardware with 64GB VRAM.
Watch out for
- MoE and quantized variants make clear logical mistakes on real tasks.
- Ranks #27 of 49 on LMArena, well behind frontier models.
GPT-5.4 ranks #9 on LMArena and serves as a solid parametric frontier model for fact-checking applications. It was included in a panel of five models for evaluating real user claims, though agreement between models was limited.
Best when: You need a frontier OpenAI model for parametric fact-checking without retrieval.
Tips
- Ranks #9 of 49 on LMArena with Elo 1470.
- Selected as a parametric frontier model for fact-checking benchmarks.
Watch out for
- Part of a panel with only limited agreement, where models frequently dissented from majority.
GLM 5.2 fixed the severe degradation issues of its predecessor, which would fail catastrophically at around 100k tokens. The evidence suggests 5.2 is usable, but long-context performance still falls off dramatically past 150k-200k tokens.
Best when: You need a Chinese model and your documents stay under 150k tokens.
Tips
- Fixed the insanity issues that plagued GLM-5.1 at 100k tokens.
Watch out for
- Long-context performance falls dramatically after 150k-200k tokens.
- Still inferior to models like Opus 4.8 which maintain quality past 500k tokens.
Frequently asked
- Which model has the largest context window for huge documents?
- Kimi K3 offers a 1 million token context window, making it suitable for EDA design, large codebases, and other workloads that would overflow smaller contexts.
- What's the cheapest model that still handles RAG well?
- DeepSeek V4 Flash is reported to be the cheapest model with acceptable RAG replies, finding the right information with few hallucinations in real-world tests.
- Do model rankings predict long-context performance?
- Not directly. GLM 5.2 works great until about 100k tokens and then degrades sharply, while Opus 4.8 maintains quality well beyond 500k tokens despite LMArena not specifically testing this.
- What should I self-host for long-context workloads?
- Gemma 4 31B in full precision handles 150k+ business contexts reasonably well, though MoE or quantized variants show clear logical mistakes on real tasks.
- How do frontier models compare for fact-checking retrievals?
- In a study of 1,000 real user claims, 67% had at least one of five frontier models (GPT-5.4, Claude Opus 4.7, Gemini 3 Pro, Gemini 3 Pro+Search, Sonar Pro) disagree with the panel majority, showing nontrivial but limited agreement.
Sources
- 1
“Ranks #14 of 17 on LMArena's overall text arena (Elo 1483), based on blind human preference votes.”
LMArena text arena · Benchmark · Jul 20, 2026 - 2
“Ranks #1 of 17 on LMArena's overall text arena (Elo 1512), based on blind human preference votes.”
LMArena text arena · Benchmark · Jul 20, 2026 - 3
“Ranks #2 of 17 on LMArena's overall text arena (Elo 1504), based on blind human preference votes.”
LMArena text arena · Benchmark · Jul 20, 2026 - 4
“So, in the past I've shared that I evaluate AI models by feeding them my ever-growing large collection of personal poems that span well over 800 poems (1000 depending on how you count) and over 250k tokens. What I do is feed it some initial prompt asking it to simply discuss what can be said when faced with this unedited, unseen collection of poetry. I ask the model to evaluate who the author is (or claims to be), what they went through in life, if there are different chronological poetic "phas…”
jorl17 · Hacker News · Jun 9, 2026 - 5
“I've been extremely impressed with DeepSeek V4 flash. We've been working on a project which can be thought of as an agent, just not for coding. So we've been building everything: agents, sub-agents, RAG, dynamic intent detection, changing models based on what's being done, etc. In our tests, DeepSeek V4-flash is the cheapest model with acceptable replies (few hallucinations, while finding the right information). It's not the cheapest one we run overall (we're actually surviving with 3B models f…”
jorl17 · Hacker News · May 22, 2026 - 6
“> From GLM 4.7 flash GLM 4.7 Flash is a 30b model that was far behind SOTA at launch, and I know that because I pay for z.ai inference and have run the model locally. Qwen and Deepseek V4 Flash have the same issue, and beg the question; are you really going to process a 64k agentic context at 450tok s? That's 2+ minutes that you spend waiting for the first token to generate! Of course nobody can sell that as competitive inference, and it only gets worse with larger models. We're talking about n…”
bigyabai · Hacker News · Jun 25, 2026 - 7
“Ranks #4 of 17 on LMArena's overall text arena (Elo 1499), based on blind human preference votes.”
LMArena text arena · Benchmark · Jul 20, 2026 - 8
“Fwiw the two models that did have access to search disagreed with each other on the bombing one: > 7.1 Model selection > Five frontier models, chosen to cover two capability surfaces: > Parametric (training-only): GPT-5.4 (OpenAI), Claude Opus 4.7 (Anthropic), Gemini 3 Pro (Google) > Retrieval-augmented: Gemini 3 Pro + Search (Google), Sonar Pro (Perplexity)”
post-it · Hacker News · May 28, 2026 - 9
“Author here. 67% (95% CI 64–70%) of 1,000 recent real user claims to a fact-checking platform had at least one of GPT-5.4, Claude Opus 4.7, Gemini 3 Pro, Gemini 3 Pro+Search, and Sonar Pro dissent from the panel majority — or no majority formed at all. Panel-level Krippendorff's α (ordinal) = 0.639, i.e. nontrivial but limited agreement. Quick context on what's in the writeup and what isn't: - What's measured: parsed-label agreement between the 5 models. Forced 4-choice (True Mostly True Mislea…”
kostaj · Hacker News · May 28, 2026 - 10
“I personally found in my work (long non-programming business-related context, 150k+) that only Gemma 4 31B full precision and full context works reasonably well. MoE Gemma 4, even non–quantised, makes clear logical mistakes or omissions. On real tasks. If I compare them on tiny tests – yes, sure, even i2 quants work great! :-)”
pbgcp2026 · Hacker News · Jul 4, 2026 - 11
“When I wanted to tinker with self-hosted models, I bought a couple of Radeon Pro V620 GPUs, because they're 32GB, still supported by current ROCm releases, and a few years newer than the similar-priced 32GB Nvidia cards (which are all EOL). They're a little faster than the old Tesla stuff, as well. 64GB is enough to run Gemma 4 31b 4-bit QAT with pretty big context at a respectable interactive speed (30+ tokens per second sustained). That said, even the old Radeon Pro stuff has gotten more expe…”
SwellJoe · Hacker News · Jul 13, 2026 - 12
“Ranks #27 of 49 on LMArena's overall text arena (Elo 1442), based on blind human preference votes.”
LMArena text arena · Benchmark · Jul 16, 2026 - 13
“Ranks #3 of 17 on LMArena's overall text arena (Elo 1499), based on blind human preference votes.”
LMArena text arena · Benchmark · Jul 20, 2026 - 14
“> But long-context is a trap, because performance still falls dramatically after 150k-200k context. I'm not sure exactly what causes the difference, but this heavily depends on the model. In my experience with Opus 4.8, I can go well over 500k and still get extremely good results. A drastically different example was GLM-5.1, which worked great until about 100k and then turned insane almost immediately. They did fix that with 5.2, though.”
InsideOutSanta · Hacker News · Jul 15, 2026
Rankings synthesized from community evidence and open benchmarks. See methodology. Not driven by vendor marketing.