Task ranking

Best LLM for Structured Data Extraction

DeepSeek V4.1 Flash ranks first for Structured Data Extraction as of November 15, 2026.[1][2][3][4] It scores 79.25% on LiveBench Data Analysis (#14 of 59), including table joining and reformatting tasks. Kimi K3 is the next-ranked alternative. Ranks #16 on LMArena with Elo 1487 and #16 on LiveBench Data Analysis with 78.73%.

About this recommendation

Updated
Evidence through
Sources
19
Revision
v139

Decision audit

Why this result

Inspect the inputs and the computed order behind the recommendation.

Models screened

24

live candidates

Evaluation feeds

5

task-weighted

Winner coverage

42%

intended feed weight

Largest provider share

2 of 8

Z.ai

Established source breadth. 13 citation families and 1 practitioner families provide attributed capability reports; 1 cautionary thread retained. The largest citation family contributes 26%.

DeepSeek: DeepSeek V4.1 Flash leads the measured task comparison with 2 independent evaluation families. Community volume, release age and usage popularity add no ranking points. This category also weights its stated price or context criteria.

Real-world track record is still developing. The recommendation can lead on independent measurements while practitioner reports accumulate.

Sources evaluated

These are the weights used in scoring. Missing measurements are omitted from each model’s average. A feed with no result for any model released in the last 60 days is stale and does not score until it catches up.

winner: DeepSeek V4.1 Flash
Evaluation feedWeightWinner resultField measured
Structured-output evaldisabled; last successful check unknownunavailable
0%
feed unavailable0/24
LiveBench Data Analysis (2026-06-25)Observed 2026-10-18 · evaluation date not supplied · deepseek-v4.1-flash-maxfailing; last successful check 2026-10-18
50%
#1424/24
Berkeley Function Callingstale
0%
no current result0/24
LMArena TextObserved 2026-11-11 · evaluation date not supplied · max effort
25%
#2624/24
price weight
25%
77/10024/24

Provider concentration

Each exact model is scored separately; provider identity is not a ranking input.

Z.ai25%
  • Z.ai2 models
  • Anthropic1 model
  • deepseek1 model
  • Meta1 model
  • Moonshot AI1 model
  • OpenAI1 model
  • Qwen1 model

Decision table

Every published model is shown in computed order. Practitioner sources are distinct community threads, not the citations repeated in the prose below.

  1. deepseek · coverage 42%

    #14 LiveBench Data Analysis · #26 LMArena Text

  2. Moonshot AI · coverage 42%

    #16 LiveBench Data Analysis · #16 LMArena Text

  3. Z.ai · coverage 42%

    #26 LMArena Text · #27 LiveBench Data Analysis

  4. Z.ai · coverage 42%

    #26 LMArena Text · #33 LiveBench Data Analysis

  5. OpenAI · coverage 42%

    #2 LiveBench Data Analysis · #22 LMArena Text

  6. Meta · coverage 42%

    #9 LiveBench Data Analysis · #9 LMArena Text

  7. Anthropic · coverage 42%

    #4 LMArena Text · #6 LiveBench Data Analysis

  8. Qwen · coverage 42%

    #24 LiveBench Data Analysis · #70 LMArena Text

Task score averages the available, task-weighted measurements. Benchmark positions compare models matched to our catalog. Models measured by at least two independent evaluator families are considered first. Missing tests, practitioner report counts, popularity and release age add no points. Coverage describes evidence breadth, not a probability of correctness. Scores within two points are shown as close under this policy; this is not a statistical confidence interval. When the leading scores are that close, the model measured by more evaluator families ranks first.

  1. 01

    DeepSeek V4.1 Flashdeepseek · $0.051 in / $0.20 out per M tokens · 1.0M context · open-weight

    DeepSeek V4.1 Flash scores 79.25% on LiveBench Data Analysis (#14 of 59), including table joining and reformatting tasks.

    Best when: Its strongest result for this task is on LiveBench Data Analysis, which measures working with tables and structured data.
    Source 1
    “Within the 59 catalog models matched to this evaluation: Scores 79.25% on LiveBench Data Analysis (#14 of 59), including table joining and reformatting tasks.”
    LiveBench Data AnalysisOpen original ↗
  2. 02

    Kimi K3Moonshot AI · $0.19 in / $16 out per M tokens · 1.0M context · open-weight

    Ranks #16 on LMArena with Elo 1487 and #16 on LiveBench Data Analysis with 78.73%.

    Best when: Its strongest result for this task is on LMArena Text, which measures general reasoning and instruction following.
    Source 2
    “Within the 147 catalog models matched to this evaluation: Ranks #16 of 147 on LMArena's overall text arena (Elo 1487), based on blind human preference votes.”
    LMArena text arenaOpen original ↗

    Watch out for

    • Chunk large prompts and reduce max_tokens to avoid Cloudflare 524 timeouts from extended reasoning phases.
      Source 4
      “What happened Two consecutive blockrun_chat calls to moonshot/kimi-k3 failed with: Date: 2026-07-22, ~morning PT, via the BlockRun MCP server in Claude Code. Repro - model: moonshot/kimi-k3 - Large prompt (~1,200-word document to edit) + max_tokens 5000–8000 - K3 is a reasoning model: it spends a long time generating reasoning_content before the answer. Wall-clock generation exceeds the gateway's origin timeout → Cloudflare 524. - A third call with a chunked prompt and `max_toke…”
  3. 03

    GLM 5.3 FlashZ.ai · $0.040 in / $0.078 out per M tokens · 1.0M context · open-weight

    Ranks #27 on LiveBench Data Analysis with 76.4%.

    Best when: Its strongest result for this task is on LMArena Text, which measures general reasoning and instruction following.
    Source 5
    “Within the 147 catalog models matched to this evaluation: Ranks #26 of 147 on LMArena's overall text arena (Elo 1475), based on blind human preference votes.”
    LMArena text arenaOpen original ↗

    Watch out for

    • Implement server-side validation with corrective retry as model produces schema-invalid JSON with missing brackets, misnested properties, and stray tokens.
      Source 6
      “Problem: A --schema run validates the final answer and reprompts once, but nothing stops a model from emitting malformed JSON twice. Scrutare's qualification of the self-hosted spark/glm-5.3-flash (vLLM behind LiteLLM, ViviDynamics/scrutare#77) lost 9 of 144 sessions to schema_violation: missing brackets, misnested properties, valid JSON followed by stray tokens. vLLM can enforce a JSON Schema at decode time through the OpenAI-compatible response_format, and nare has no way to ask f…”
      Source 7
      “Problem: Self-hosted models produce schema-invalid final answers often enough to fail protocol qualification. In the spark/glm-5.3-flash qualification (PR #77, evaluations/reports/2026-10-14-glm-qualification), 9 of 144 sessions ended with malformed or misnested JSON that survived nare's single reprompt, and one g1 cell was refused for {"findings": []} followed by stray tokens and U+FFFD. At that rate a clean 36-cell schedule is close to impossible, and no profile setting fixes a malf…”
    • Watch for generation loops requiring LiteLLM abortion due to repeating output.
      Source 8
      “Problem: spark/glm-5.3-flash has failed four frozen schedules in a row: g2 (PR #77), s1 (PR #80), s2 (PR #81) and c1 (PR #85). Each fix removed one failure mode and exposed another: - schema-constrained output removed malformed answers; - larger budgets removed overruns; - the output cap removed runaway generations. What remains is the model itself: - citations past a file's end; - a placeholder content hash; - a generation loop that LiteLLM aborted ("The model is repeating the sa…”
    • Expect model to narrate analysis instead of returning required JSON despite explicit instructions.
      Source 9
      “Problem Cited research can fail without returning any usable suggestions because some models spend their entire response budget narrating analysis or planned searches instead of producing the required JSON. Example from glm-5.3-flash: > Let me analyze this task: > 1. We have a member of parliament... > 2. The recorded school name is... > ... > Let me start with these. The current research prompt already says Return only JSON, but only after instructing the model to search. That is not st…”
    • Account for 14.4% rate of completely missing checklist output in evaluations.
      Source 10
      “Part of #45 Question glm-5.3-flash returned no checklist at all in 14.4% of evaluations, and a missing checklist currently disables the gate (#44). vLLM supports guided decoding against a JSON schema; the Anthropic path has structured outputs. Make the VLM call on OpenAI-compatible providers constrain the response to the evaluation schema so a structurally invalid response cannot occur, and confirm the Anthropic path is unaffected. Records on resolution: the schema used, and the missing-…”
  4. 04

    GLM 5.2Z.ai · $0.007 in / $6.80 out per M tokens · 1.0M context · open-weight

    GLM 5.2 ranks #26 of 147 on LMArena's overall text arena (Elo 1475), based on blind human preference votes.

    Best when: Its strongest result for this task is on LMArena Text, which measures general reasoning and instruction following.
    Source 11
    “Within the 147 catalog models matched to this evaluation: Ranks #26 of 147 on LMArena's overall text arena (Elo 1475), based on blind human preference votes.”
    LMArena text arenaOpen original ↗
  5. 05

    GPT-6.1 SolOpenAI · $2 in / $10 out per M tokens · 1.1M context

    GPT-6.1 Sol scores 82.66% on LiveBench Data Analysis (#2 of 59), including table joining and reformatting tasks.

    Best when: Its strongest result for this task is on LiveBench Data Analysis, which measures working with tables and structured data.
    Source 12
    “Within the 59 catalog models matched to this evaluation: Scores 82.66% on LiveBench Data Analysis (#2 of 59), including table joining and reformatting tasks.”
    LiveBench Data AnalysisOpen original ↗

    No practitioner reports are linked yet, so no cautions are listed. Absence of reports is not evidence of reliability.

  6. 06

    Muse Spark 1.3Meta · $1.25 in / $4.25 out per M tokens · 1.0M context

    Ranks #9 on LiveBench Data Analysis with 79.57% for table joining and reformatting tasks, and #9 on LMArena with Elo 1497.

    Best when: Use for data analysis tasks where LiveBench table performance is a priority.
    Source 13
    “Within the 59 catalog models matched to this evaluation: Scores 79.57% on LiveBench Data Analysis (#9 of 59), including table joining and reformatting tasks.”
    LiveBench Data AnalysisOpen original ↗

    No practitioner reports are linked yet, so no cautions are listed. Absence of reports is not evidence of reliability.

  7. 07

    Claude Opus 5.5Anthropic · $4 in / $20 out per M tokens · 1M context

    Ranks #4 on LMArena overall text arena with Elo 1507.

    Best when: Use response_format with JSON schema instead of tool calling for structured extraction workflows.
    Source 14
    “Answer Output format: use a strict JSON schema response_format, as Velora does today, not tool calling. OpenRouter already routes response_format requests only to endpoints that support it. Forcing a tool call on current Claude models either skips reasoning (Haiku 5.5) or is rejected (Opus/Sonnet 5.5). Strict-mode enforcement varies by provider, so keep the server-side validation and the single corrective retry. Model: keep openai/gpt-6-luna at reasoning.effort: "low" ($0.10…”
    prabhuakshayOpen original ↗

    More tips

    • Deploy for geographic label extraction from maps where subagent parsing is required.
      Source 15
      “How a map on a café wall turned into an app that reads, maps and translates all 387 labels of an 1819 French map of the Bosphorus, built with Claude Opus 5.5 subagents in an afternoon. I wrote up where this came from and how I built it: https: cahidarda.com articles bosphore-1819”

    Watch out for

    • Watch for final messages that are not JSON in multi-turn review tasks.
      Source 16
      “Objective A targeted reviewer launched by agent-launcher always returns a parseable answer, or leaves enough evidence to say why it did not. Intent The saga-review-corpus pilot (saga-review-corpus#3, on infiquetra-agent-plugins b04ff8b) stopped twice on the same tuning case because the primary Claude reviewer's final message was not JSON: "exit": 3, "error": "the final message is not JSON" (Opus 5.5 high, 12 turns, 9,633 output tokens, $0.73). One of the first six reviews failed this…”
  8. 08

    Qwen3.8 27BQwen · $0.15 in / $1.88 out per M tokens · 1.0M context · open-weight

    Ranks #24 on LiveBench Data Analysis with 76.59%.

    Best when: Use for data analysis tasks where LiveBench table performance is acceptable.
    Source 17
    “Within the 59 catalog models matched to this evaluation: Scores 76.59% on LiveBench Data Analysis (#24 of 59), including table joining and reformatting tasks.”
    LiveBench Data AnalysisOpen original ↗

    Watch out for

    • Validate JSON output rigorously as model returns malformed responses for schema-constrained tasks.
      Source 18
      “Follow-up to #47 (local-model testing on v0.3.0). Related to the fact-sheet issue (#61); this can ship independently. Problem Local models returned malformed responses for the investigation step: - qwen3.8:27b: "Invalid investigation response: Investigation payload_json must encode valid JSON: a plan object or an evidence request array" - Drafting with qwen3:8b exhausted its 3 calls on schema details ("Accepted inputs need a label and lowercase sha1 or sha256"). These are format failures, no…”
    • Budget ~46 seconds per dialogue for extraction tasks on A100 80GB PCIe via ollama.
      Source 19
      “抽出の所要時間(壁時計) run_extraction.py は所要時間を記録しないため、実行ループのログの開始・終了時刻から求めた。対象は全 47 対話、GPU は A100 80GB PCIe(ollama)。 | モデル | 開始 | 終了 | 所要時間 | 1対話あたり | |---|---|---|---|---| | gemma4:e4b(ollama) | 10:03:24 | 11:50:34 | 1時間47分10秒 | 約 137 秒 | | gpt-oss:20b(ollama) | 11:50:34 | 12:03:09 | 12分35秒 | 約 16 秒 | | qwen3.8:27b(ollama) | 12:03:09 | 12:39:17 | 36分08秒 | 約 46 秒 | | gpt-6-luna(OpenAI API) | 記録なし | 10:12:34(保存時刻) | 不明 | test.json の 8 対話では約 7 秒 | - 3 モデルは同じ GPU で順番に実行した(並行実行なし)。gpt-6-luna の抽出は gemma…”

Frequently asked

What is the best model for Structured Data Extraction?
DeepSeek V4.1 Flash ranks first as of November 15, 2026. DeepSeek V4.1 Flash scores 79.25% on LiveBench Data Analysis (#14 of 59), including table joining and reformatting tasks.[1]
What is the best alternative to DeepSeek V4.1 Flash?
Kimi K3 ranks second. Ranks #16 on LMArena with Elo 1487 and #16 on LiveBench Data Analysis with 78.73%.[2][3][4]
Can I self-host the top pick for Structured Data Extraction?
Yes. DeepSeek V4.1 Flash is open-weight, so it can run on your own hardware as well as through hosted routes.[1]

Sources

  1. 1

    “Within the 59 catalog models matched to this evaluation: Scores 79.25% on LiveBench Data Analysis (#14 of 59), including table joining and reformatting tasks.”

    LiveBench Data Analysis · Benchmark · Oct 18, 2026
  2. 2

    “Within the 147 catalog models matched to this evaluation: Ranks #16 of 147 on LMArena's overall text arena (Elo 1487), based on blind human preference votes.”

    LMArena text arena · Benchmark · Nov 11, 2026
  3. 3

    “Within the 59 catalog models matched to this evaluation: Scores 78.73% on LiveBench Data Analysis (#16 of 59), including table joining and reformatting tasks.”

    LiveBench Data Analysis · Benchmark · Oct 18, 2026
  4. 4

    “What happened Two consecutive blockrun_chat calls to moonshot/kimi-k3 failed with: Date: 2026-07-22, ~morning PT, via the BlockRun MCP server in Claude Code. Repro - model: moonshot/kimi-k3 - Large prompt (~1,200-word document to edit) + max_tokens 5000–8000 - K3 is a reasoning model: it spends a long time generating reasoning_content before the answer. Wall-clock generation exceeds the gateway's origin timeout → Cloudflare 524. - A third call with a chunked prompt and `max_toke…”

    VickyXAI · GitHub · Jul 22, 2026
  5. 5

    “Within the 147 catalog models matched to this evaluation: Ranks #26 of 147 on LMArena's overall text arena (Elo 1475), based on blind human preference votes.”

    LMArena text arena · Benchmark · Nov 11, 2026
  6. 6

    “Problem: A --schema run validates the final answer and reprompts once, but nothing stops a model from emitting malformed JSON twice. Scrutare's qualification of the self-hosted spark/glm-5.3-flash (vLLM behind LiteLLM, ViviDynamics/scrutare#77) lost 9 of 144 sessions to schema_violation: missing brackets, misnested properties, valid JSON followed by stray tokens. vLLM can enforce a JSON Schema at decode time through the OpenAI-compatible response_format, and nare has no way to ask f…”

    Jason733i · GitHub · Oct 15, 2026
  7. 7

    “Problem: Self-hosted models produce schema-invalid final answers often enough to fail protocol qualification. In the spark/glm-5.3-flash qualification (PR #77, evaluations/reports/2026-10-14-glm-qualification), 9 of 144 sessions ended with malformed or misnested JSON that survived nare's single reprompt, and one g1 cell was refused for {"findings": []} followed by stray tokens and U+FFFD. At that rate a clean 36-cell schedule is close to impossible, and no profile setting fixes a malf…”

    Jason733i · GitHub · Oct 15, 2026
  8. 8

    “Problem: spark/glm-5.3-flash has failed four frozen schedules in a row: g2 (PR #77), s1 (PR #80), s2 (PR #81) and c1 (PR #85). Each fix removed one failure mode and exposed another: - schema-constrained output removed malformed answers; - larger budgets removed overruns; - the output cap removed runaway generations. What remains is the model itself: - citations past a file's end; - a placeholder content hash; - a generation loop that LiteLLM aborted ("The model is repeating the sa…”

    Jason733i · GitHub · Oct 25, 2026
  9. 9

    “Problem Cited research can fail without returning any usable suggestions because some models spend their entire response budget narrating analysis or planned searches instead of producing the required JSON. Example from glm-5.3-flash: > Let me analyze this task: > 1. We have a member of parliament... > 2. The recorded school name is... > ... > Let me start with these. The current research prompt already says Return only JSON, but only after instructing the model to search. That is not st…”

    Mappboy · GitHub · Oct 12, 2026
  10. 10

    “Part of #45 Question glm-5.3-flash returned no checklist at all in 14.4% of evaluations, and a missing checklist currently disables the gate (#44). vLLM supports guided decoding against a JSON schema; the Anthropic path has structured outputs. Make the VLM call on OpenAI-compatible providers constrain the response to the evaluation schema so a structurally invalid response cannot occur, and confirm the Anthropic path is unaffected. Records on resolution: the schema used, and the missing-…”

    kreuzhofer · GitHub · Sep 2, 2026
  11. 11

    “Within the 147 catalog models matched to this evaluation: Ranks #26 of 147 on LMArena's overall text arena (Elo 1475), based on blind human preference votes.”

    LMArena text arena · Benchmark · Nov 11, 2026
  12. 12

    “Within the 59 catalog models matched to this evaluation: Scores 82.66% on LiveBench Data Analysis (#2 of 59), including table joining and reformatting tasks.”

    LiveBench Data Analysis · Benchmark · Oct 18, 2026
  13. 13

    “Within the 59 catalog models matched to this evaluation: Scores 79.57% on LiveBench Data Analysis (#9 of 59), including table joining and reformatting tasks.”

    LiveBench Data Analysis · Benchmark · Oct 18, 2026
  14. 14

    “Answer Output format: use a strict JSON schema response_format, as Velora does today, not tool calling. OpenRouter already routes response_format requests only to endpoints that support it. Forcing a tool call on current Claude models either skips reasoning (Haiku 5.5) or is rejected (Opus/Sonnet 5.5). Strict-mode enforcement varies by provider, so keep the server-side validation and the single corrective retry. Model: keep openai/gpt-6-luna at reasoning.effort: "low" ($0.10…”

    prabhuakshay · GitHub · Oct 16, 2026
  15. 15

    “How a map on a café wall turned into an app that reads, maps and translates all 387 labels of an 1819 French map of the Bosphorus, built with Claude Opus 5.5 subagents in an afternoon. I wrote up where this came from and how I built it: https: cahidarda.com articles bosphore-1819”

    cahid_arda · Hacker News · Sep 24, 2026
  16. 16

    “Objective A targeted reviewer launched by agent-launcher always returns a parseable answer, or leaves enough evidence to say why it did not. Intent The saga-review-corpus pilot (saga-review-corpus#3, on infiquetra-agent-plugins b04ff8b) stopped twice on the same tuning case because the primary Claude reviewer's final message was not JSON: "exit": 3, "error": "the final message is not JSON" (Opus 5.5 high, 12 turns, 9,633 output tokens, $0.73). One of the first six reviews failed this…”

    namredips · GitHub · Oct 16, 2026
  17. 17

    “Within the 59 catalog models matched to this evaluation: Scores 76.59% on LiveBench Data Analysis (#24 of 59), including table joining and reformatting tasks.”

    LiveBench Data Analysis · Benchmark · Oct 18, 2026
  18. 18

    “Follow-up to #47 (local-model testing on v0.3.0). Related to the fact-sheet issue (#61); this can ship independently. Problem Local models returned malformed responses for the investigation step: - qwen3.8:27b: "Invalid investigation response: Investigation payload_json must encode valid JSON: a plan object or an evidence request array" - Drafting with qwen3:8b exhausted its 3 calls on schema details ("Accepted inputs need a label and lowercase sha1 or sha256"). These are format failures, no…”

    Crimson3076 · GitHub · Nov 3, 2026
  19. 19

    [Exp] 前処理モデルの候補(gemma4:e4b / gpt-oss:20b / qwen3.8:27b / gpt-6-luna)の抽出性能を gpt-4o と比べる

    Original-language excerpt:“抽出の所要時間(壁時計) run_extraction.py は所要時間を記録しないため、実行ループのログの開始・終了時刻から求めた。対象は全 47 対話、GPU は A100 80GB PCIe(ollama)。 | モデル | 開始 | 終了 | 所要時間 | 1対話あたり | |---|---|---|---|---| | gemma4:e4b(ollama) | 10:03:24 | 11:50:34 | 1時間47分10秒 | 約 137 秒 | | gpt-oss:20b(ollama) | 11:50:34 | 12:03:09 | 12分35秒 | 約 16 秒 | | qwen3.8:27b(ollama) | 12:03:09 | 12:39:17 | 36分08秒 | 約 46 秒 | | gpt-6-luna(OpenAI API) | 記録なし | 10:12:34(保存時刻) | 不明 | test.json の 8 対話では約 7 秒 | - 3 モデルは同じ GPU で順番に実行した(並行実行なし)。gpt-6-luna の抽出は gemma…”

    aidayu · GitHub · Oct 23, 2026

Rankings synthesized from community evidence and open benchmarks. See methodology. Not driven by vendor marketing.