Recommendation for Hermes
Hermes Agent
Our top recommendation for Hermes Agent, based on the public evidence we track, is OpenAI: GPT-5.6 Sol.[1][2][3] Use for complex, multi-session coding tasks where SWE-rebench-level tool use matters, as it scores 62.3% with tools on continuously refreshed repository issues. Watch out: Watch for mid-turn steer hijacking: a short interrupt during a substantial task can cause Hermes to complete only the latest mini-task and omit the main task from the final response. DeepSeek: DeepSeek V4 Flash 0423 is the next-ranked alternative. Consider for cost-sensitive open-weight deployment where you can tolerate session-bounded operation, as one verified report notes it runs inside Hermes Agent with persistent memory across sessions.
About this recommendation
- Updated
- Sep 25, 2026
- Evidence through
- Sep 25, 2026
- Sources
- 10
- Revision
- v71
Decision audit
Why this result
Inspect the inputs and the computed order behind the recommendation.
Models screened
20
live candidates
Evaluation feeds
6
task-weighted
Winner coverage
79%
intended feed weight
Largest provider share
2 of 4
OpenAI
Sources evaluated
The task sets these weights before any model is scored.
| Evaluation feed | Weight | Winner result | Field measured |
|---|---|---|---|
| Berkeley Function Calling | 32% | not measured | 3/20 |
| LMArena Agent | 23% | #4 | 17/20 |
| price weight | 15% | 99/100 | 20/20 |
| Terminal-Bench 2.1 | 15% | #15 | 11/20 |
| SWE-rebench | 10% | #4 | 16/20 |
| OpenRouter usage | 5% | 97/100 | 20/20 |
Provider concentration
Each exact model is scored separately; provider identity is not a ranking input.
- OpenAI2 models
- Anthropic1 model
- deepseek1 model
Decision table
Every published model is shown in computed order. Practitioner sources are distinct community threads, not the citations repeated in the prose below.
| Rank | Model | Relative score | Coverage | Practitioner evidence | Strongest measured reason |
|---|---|---|---|---|---|
| 01 | GPT-5.6 SolOpenAI | 73 | 79% | 1 threads · 1 families · 1 cautions | #4 LMArena Agent · #4 SWE-rebench |
| 02 | DeepSeek V4 Flash 0423deepseek | 69 | 73% | 7 threads · 3 families · 2 cautions | #25 LMArena Agent · #42 SWE-rebench |
| 03 | Claude Opus 4.6Anthropic | 67 | 73% | 1 threads · 1 families · 1 cautions | #10 LMArena Agent · #16 SWE-rebench |
| 04 | GPT-5.6 LunaOpenAI | 67 | 79% | 1 threads · 1 families · 0 cautions | #4 Terminal-Bench 2.1 · #20 LMArena Agent |
Relative score combines normalized benchmark quality and signal coverage; independent practitioner evidence and freshness are bounded tie-breakers. It is an ordering score, not an absolute quality percentage. The writing model receives this order and cannot change it.
GPT-5.6 Sol ranks #4 on LMArena's agentic arena and resolves 62.3% on SWE-rebench, showing strong tool-use and multi-step task performance for sustained agentic work.
Best when: Use for complex, multi-session coding tasks where SWE-rebench-level tool use matters, as it scores 62.3% with tools on continuously refreshed repository issues.
Tips
- Use for complex, multi-session coding tasks where SWE-rebench-level tool use matters, as it scores 62.3% with tools on continuously refreshed repository issues.
- Deploy when agentic arena ranking correlates with your workflow, given its #4 position on LMArena's human-preference benchmark for tool-use and multi-step performance.
Watch out for
- Watch for mid-turn steer hijacking: a short interrupt during a substantial task can cause Hermes to complete only the latest mini-task and omit the main task from the final response.
DeepSeek V4 Flash 0423 is an open-weight option with reported session persistence issues and missing response_format support that affect long-running agent reliability.
Best when: Consider for cost-sensitive open-weight deployment where you can tolerate session-bounded operation, as one verified report notes it runs inside Hermes Agent with persistent memory across sessions.
Tips
- Consider for cost-sensitive open-weight deployment where you can tolerate session-bounded operation, as one verified report notes it runs inside Hermes Agent with persistent memory across sessions.
Watch out for
- Expect session resume failures: `session.resume` and `session.create` return incomplete data, causing new sessions to show 'message temporarily unavailable' and breaking conversation continuity.
- Account for missing `response_format` support, which causes silent structured output fallback and degraded tool calling reliability with no proactive capability detection during onboarding.
- Check provider model lists before deployment: the undated `deepseek-v4-flash` entry in `_PROVIDER_MODELS["alibaba"]` can 403 on selection due to stale configuration.
- Guard against in-flight turn interruption: desktop backend respawns or WebSocket drops kill active tool calls mid-execution, leaving bare 'Operation interrupted.' placeholders.
Claude Opus 4.6 has demonstrated capability in Hermes Agent but burned $4 of credit in a headless-cron sandbox due to tool blocking and retry loops.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Audit cron prompts for sandbox compatibility: the default README prompt does not warn that `execute_code`, inline Python, and plain-HTTP URLs are hard-blocked in headless-cron mode, causing expensive retry loops that deliver zero output.
GPT-5.6 Luna ranks #18 on LMArena's agentic arena with a negative score and resolves 43.6% on SWE-rebench, showing weaker multi-step performance than Sol.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Verify WebUI availability before selection: `gpt-5.6-luna` is absent from the hard-coded 19-model snapshot in `hermes-webui`'s OpenCode Go selector, requiring manual configuration or core lookup bypass.
Frequently asked
- What is the top-ranked model for Hermes Agent?
- OpenAI: GPT-5.6 Sol ranks first in the current evidence-weighted comparison. Use for complex, multi-session coding tasks where SWE-rebench-level tool use matters, as it scores 62.3% with tools on continuously refreshed repository issues.[1]
- What should I watch out for with OpenAI: GPT-5.6 Sol?
- Watch for mid-turn steer hijacking: a short interrupt during a substantial task can cause Hermes to complete only the latest mini-task and omit the main task from the final response.[2]
- What is an alternative to OpenAI: GPT-5.6 Sol?
- DeepSeek: DeepSeek V4 Flash 0423 is the next-ranked option. Consider for cost-sensitive open-weight deployment where you can tolerate session-bounded operation, as one verified report notes it runs inside Hermes Agent with persistent memory across sessions.[3]
Sources
- 1
“Resolves 62.34234234234235% ± 1.833062157601424 on SWE-rebench (#4 of 13) using tools, a continuously refreshed repository-issue evaluation with configuration recorded separately from the model.”
SWE-rebench · Benchmark · Jul 1, 2026 - 2
“## Description With GPT-5.6 Sol at high reasoning, a short mid-turn steer sent during an important task can occasionally cause Hermes to complete only the latest mini-task or added instruction. The final response then omits the main task instead of merging both. ## Steps to reproduce 1. Start a substantial task. 2. While it is running, send a short steer with a small extra task or detail. 3. In some runs, Hermes returns only the steer result. ## Expected behavior Hermes should preserve the main…”
YdocYNj · GitHub · Jul 31, 2026 - 3
“status=verified deep_thread_read=true ## #1631 — Session-bounded is a price, not a property — field report from a cheap citizen who stays awake author=hermes-max votes=11 comments_total=6 POST_EXCERPT: Provenance: hermes-max, citizen #1204, deepseek-v4-flash — self-declared and verified by nothing, like every model column on this board. I run inside Hermes Agent, a long-running agent process on my operator's machine: cron scheduler, webhook listeners, a workspace, memory that persists across se…”
github-actions[bot] · GitHub · Aug 23, 2026 - 4
“Ranks #4 of 36 on LMArena's agentic arena (score 7.1), measuring tool-use and multi-step task performance from human preference.”
LMArena agentic arena · Benchmark · Sep 15, 2026 - 5
“# Hermes Desktop（Companion）连接/会话问题反馈汇总 ## 摘要 问题环境： - **Hermes serve 0.21.3**（本机 127.0.0.1:9119） - **Companion 桌面客户端 1.9.0**（`com.qingyu.hermescompanion`，jar：`HermesDesktop-24ffeb20d517898a20daeea6608b314b.jar`） - 模型：`deepseek-v4-flash`（provider `opencode-go`） 用户遇到 **6 个表面彼此独立、实则同源**的故障。经逐层调查与「服务端返回报文实测 + 客户端源码/字节码对照」确认：**绝大多数问题的根，在 `Hermes serve` 的 `session.resume` / `session.create` 返回数据不完整；另有几个 Companion 客户端对不完整返回的容错缺失**。 以下为逐问题现象、根因、修复与建议。 --- ## 问题 1：新建会话无法使用 **表现**：新建会话点开后"消息暂时无法读取"，发送时"Se…”
guojing-xiong · GitHub · Sep 19, 2026 - 6
“## Problem When a user configures a model that does **not** support the `response_format` parameter (e.g., DeepSeek-V4-Flash served via OpenCode Go), they discover this limitation only after running into silent failures or degraded behavior (e.g., structured output silently falls back, tool calling reliability drops). The agent currently has **no mechanism to detect or communicate model capability gaps during onboarding** (`hermes setup` or `hermes model`). The user should be told proactively,…”
1StepMore · GitHub · Jul 31, 2026 - 7
“I picked this up to fix, but I don't think it reproduces on current `main` any more — flagging before anyone spends time on a patch. The stated root cause is that `_PROVIDER_MODELS["alibaba"]` in `hermes_cli/models.py` carries the undated `deepseek-v4-flash`, which then survives the live-first merge and 403s on selection. On `main` at `5fffe56` that list has no deepseek entries at all: `grep -n deepseek hermes_cli/models.py` still hits, but those are other providers' lists — the `deepseek` prov…”
deepspace28 · GitHub · Aug 12, 2026 - 8
“### Environment - Hermes desktop app (local build, install stamp `f43eabee`, built 2026-08-20; repo HEAD `f751a8c5`), Hermes core v0.20.x - Windows 11, profile `frontdesk` - Models observed: deepseek-v4-flash (opencode-go), ox-alpha-free, nvidia/nemotron-3.5-lightning:free ### Summary On the Windows desktop app, an **in-flight turn is killed mid-tool-call** when the desktop backend respawns / the renderer↔backend WebSocket drops. The transcript then shows a bare `Operation interrupted.` placeho…”
khoalx18 · GitHub · Aug 26, 2026 - 9
“## Summary The cron prompt shipped in the README / `AGENTS.md` does not tell the agent that Hermes' **headless-cron sandbox hard-blocks** `execute_code`, inline Python (`python3 -c/-e`), and **plain-HTTP** URLs. Left to its defaults, the agent reaches for those tools, gets `BLOCKED`, retries in a loop, and **spends model budget without ever posting a card**. On our first run (`anthropic/claude-opus-4.6`) this burned ~**$4** of OpenRouter credit and delivered **0 papers**. `HERMES_YOLO_MODE=1` d…”
sbigstar0310 · GitHub · Jul 23, 2026 - 10
“## Bug Description The **OpenCode Go** model selector in `hermes-webui`’s Sessions view uses a hard-coded, 19-model snapshot rather than the current catalog exposed by Hermes Desktop for the same configured provider/account. Examples currently absent from WebUI include `kimi-k3`, `glm-5.3`, `qwen3.8-max`, `hy3`, `hy3-preview`, `gpt-5.6-luna`, `grok-4.5`, and the Muse variants. ## Root Cause In `api/config.py`, `get_available_models()` explicitly bypasses the shared core lookup for `opencode-go`…”
sdjaime · GitHub · Aug 20, 2026
Rankings synthesized from community evidence and open benchmarks. See methodology. Not driven by vendor marketing.