updatedHumanity’s Last Exam
What changed · 1 fields
- summary
Humanity’s Last Exam evaluates advanced knowledge and reasoning across a broad range of academic subjects using expert-written questions. It includes text and multimodal problems designed to challenge frontier AI systems. Results identify whether the full exam or a text-only subset was used. → Tests whether a model can work through difficult expert-level questions, helping you choose an assistant for demanding research and technical analysis. Full-exam and text-only results are separate, and exam accuracy does not measure client-project completion.
updatedHumanity’s Last Exam
What changed · 1 fields
- summary
Humanity’s Last Exam evaluates advanced knowledge and reasoning across a broad range of academic subjects using expert-written questions. It includes text and multimodal problems designed to challenge frontier AI systems. Results identify whether the full exam or a text-only subset was used. → Tests whether a model can work through difficult expert-level questions, helping you choose an assistant for demanding research and technical analysis. Full-exam and text-only results are separate, and exam accuracy does not measure client-project completion.
updatedHumanity’s Last Exam
What changed · 1 fields
- summary
Humanity’s Last Exam evaluates advanced knowledge and reasoning across a broad range of academic subjects using expert-written questions. It includes text and multimodal problems designed to challenge frontier AI systems. Results identify whether the full exam or a text-only subset was used. → Tests whether a model can work through difficult expert-level questions, helping you choose an assistant for demanding research and technical analysis. Full-exam and text-only results are separate, and exam accuracy does not measure client-project completion.
updatedDeepSWE
What changed · 1 fields
- summary
DeepSWE evaluates coding agents on original, long-horizon software engineering tasks. It tests whether an agent can understand a codebase, make changes, and complete engineering work beyond short coding questions. → Tests whether an agent can understand a codebase and complete substantial engineering changes, helping you assess which model could take on more development work while you focus on requirements and review.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedAutomationBench-AA
What changed · 1 fields
- summary
AutomationBench-AA evaluates agents working across simulated business applications. Its score measures the share of task objectives completed without guardrail violations. This objective-level metric is separate from the strict task-completion score on Zapier’s hosted AutomationBench leaderboard. → Tests how much of a business workflow an agent can complete without breaking its rules, helping you identify opportunities to reduce manual work across apps. Scores measure objectives completed, not the percentage of entire workflows successfully finished.
updatedMCP-Atlas
What changed · 1 fields
- summary
MCP-Atlas evaluates real-world tool use through the Model Context Protocol. It tests tool discovery, choosing correct arguments, recovering from errors, and combining tool outputs to complete multi-step workflows. The stored results below cover the public subset under the April 2026 protocol. → Tests whether an agent can find and use the right connected tools, helping you assess its ability to complete workflows across your business systems with less manual coordination. These results cover the public April 2026 subset.
updatedOSWorld
What changed · 1 fields
- summary
OSWorld evaluates computer-use agents on practical tasks in a desktop environment. It tests whether agents can interact with applications and complete multi-step work through a graphical interface. Results below are restricted to the stored 2.1 full-set evaluation. → Tests whether an agent can carry out work through desktop applications, helping you assess which computer tasks you might delegate instead of doing every click yourself. Full-set, offline-subset, partial-credit and strict-success results measure different things.
updatedBrowseComp
What changed · 1 fields
- summary
BrowseComp evaluates web research agents on questions with hard-to-find but verifiable answers. It tests persistent browsing, search, and the ability to connect information from multiple sources. Tool access, search budgets, and context management can differ across published comparison reports. → Tests whether an agent can find hard-to-locate, verifiable information online, helping you choose a model for client research that would otherwise require persistent manual searching. Search tools and budgets differ across evaluation setups.
updatedGPQA Diamond
What changed · 1 fields
- summary
GPQA Diamond tests graduate-level scientific knowledge and reasoning with difficult multiple-choice questions in biology, chemistry, and physics. Diamond is the 198-question subset selected for high-quality expert agreement. The results here use the independently reported Diamond evaluation. → Tests difficult scientific questions in biology, chemistry and physics, helping you choose a model to support technical research and check complex explanations. Results cover the 198-question Diamond subset, not hands-on laboratory work.
updatedDeepSWE
What changed · 1 fields
- summary
DeepSWE evaluates coding agents on original, long-horizon software engineering tasks. It tests whether an agent can understand a codebase, make changes, and complete engineering work beyond short coding questions. → Tests whether an agent can understand a codebase and complete substantial engineering changes, helping you assess which model could take on more development work while you focus on requirements and review.
updatedBrowseComp
What changed · 1 fields
- summary
BrowseComp evaluates web research agents on questions with hard-to-find but verifiable answers. It tests persistent browsing, search, and the ability to connect information from multiple sources. Tool access, search budgets, and context management can differ across published comparison reports. → Tests whether an agent can find hard-to-locate, verifiable information online, helping you choose a model for client research that would otherwise require persistent manual searching. Search tools and budgets differ across evaluation setups.
updatedAutomationBench
What changed · 1 fields
- summary
AutomationBench tests agents on realistic business workflows across applications and APIs, covering sales, marketing, operations, support, finance, and HR. It evaluates whether agents can coordinate multi-step work and complete tasks. Zapier’s hosted leaderboard measures strict task completion on a private held-out set. → Tests whether an agent can finish business workflows across apps and APIs, helping you assess what sales, support or operations work you could automate to spend more time with clients. The hosted Zapier track measures complete tasks on a private test set.
updatedGDPval-AA
What changed · 1 fields
- summary
GDPval-AA evaluates models on professional knowledge-work tasks from OpenAI’s GDPval dataset, including creating documents, spreadsheets, slides, and other work products. Agents use tools through the Stirrup framework. Results are Elo ratings from blind pairwise comparisons, rather than percentage accuracy. → Compares the quality of professional work products such as documents, spreadsheets and slides, helping you choose a model for preparing client deliverables. Elo is a relative quality rating, not a completion percentage or a measure of time saved.
updatedSWE-bench Verified
What changed · 1 fields
- summary
SWE-bench evaluates software engineering agents on resolving real issues in existing code repositories. The Verified subset contains human-validated tasks. Scores report the proportion of issues resolved; agent harnesses and evaluation settings are retained with each published model result. → Tests whether an agent can fix real GitHub issues, helping you choose a model for bug fixes and maintenance so you can focus on reviewing changes and serving clients. The Verified set contains 500 human-validated issues; agent setups remain separate.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedBrowseComp
What changed · 1 fields
- summary
BrowseComp evaluates web research agents on questions with hard-to-find but verifiable answers. It tests persistent browsing, search, and the ability to connect information from multiple sources. Tool access, search budgets, and context management can differ across published comparison reports. → Tests whether an agent can find hard-to-locate, verifiable information online, helping you choose a model for client research that would otherwise require persistent manual searching. Search tools and budgets differ across evaluation setups.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedAutomationBench
What changed · 1 fields
- summary
AutomationBench tests agents on realistic business workflows across applications and APIs, covering sales, marketing, operations, support, finance, and HR. It evaluates whether agents can coordinate multi-step work and complete tasks. Zapier’s hosted leaderboard measures strict task completion on a private held-out set. → Tests whether an agent can finish business workflows across apps and APIs, helping you assess what sales, support or operations work you could automate to spend more time with clients. The hosted Zapier track measures complete tasks on a private test set.
updatedLifeVal
What changed · 1 fields
- summary
LifeVal evaluates personal assistants on everyday responsibilities, including caregiving, household administration, shopping, education, personal finances, and accessing government services. Scores are unweighted average rewards across 169 tasks, expressed as percentages, rather than a strict task-completion rate. The launch results cover five OpenAI models. → Tests help with everyday responsibilities such as shopping, household administration and personal finances, helping you assess an assistant for reducing routine life admin. Scores are average rewards across 169 tasks, not the percentage of tasks fully completed.
updatedMyPCBench
What changed · 1 fields
- summary
MyPCBench evaluates assistants on 184 tasks across a personal desktop with 17 simulated web apps. Tasks involve personal records, travel, shopping, calendars, and coordinating work across applications. These results use the canonical computer-and-bash setup with a 100-step limit and report the percentage of tasks where every rubric item passes. → Tests whether an assistant can complete personal desktop tasks across calendars, travel, shopping and records, helping you assess what routine coordination you could delegate. A task passes only when every rubric item is satisfied within the 100-step setup.
updatedAppWorld
What changed · 1 fields
- summary
AppWorld evaluates agents completing everyday tasks across nine simulated apps through code and APIs. Tasks include coordinating messages, shopping, and personal information. Scores here measure Task Goal Completion on the Test-Challenge split: the percentage of tasks for which all checks pass. Agent methods are retained with each result. → Tests whether an agent can complete everyday tasks across simulated apps using code and APIs, helping you choose an assistant for reducing repetitive app-to-app work. Test-Challenge scores require all task checks to pass.
updatedGaia2
What changed · 1 fields
- summary
Gaia2 evaluates assistants coordinating tasks across simulated email, calendars, shopping, contacts, and other apps. It tests execution, search, adaptation, ambiguity handling, and time constraints. This ranking uses the Gaia2-CLI leaderboard with the OpenClaw 2026.4.1 harness and reports overall pass@1. The older vanilla, noise, and agent-to-agent results are separate evaluations. → Tests whether an assistant can coordinate work across email, calendars, shopping and other apps, helping you assess how much routine planning and execution you could hand off. These results use the CLI and OpenClaw setup; other setups are separate.
updatedπ-Bench
What changed · 1 fields
- summary
π-Bench evaluates assistants across 100 multi-turn tasks in persistent workspaces. It tests whether an agent notices unstated preferences and requirements, asks useful questions, and carries context across sessions. Scores here use overall Proactivity (PROC), the share of hidden intents resolved proactively. Completeness is a separate metric and is retained with the source results. → Tests whether an assistant notices needs you have not explicitly stated, helping you choose one that needs less repeated prompting and remembers context across sessions. The score measures hidden intents handled proactively, not overall task completion.
updatedτ²-bench
What changed · 1 fields
- summary
τ²-bench evaluates conversational agents completing customer-service tasks in airline, retail, and telecom environments. Agents use tools, follow policies, and coordinate with a simulated user who can also act. Scores here are overall pass@1 from the Standard leaderboard, using GPT-5.2 as the user simulator. Custom, legacy, voice, and banking evaluations are separate. → Tests whether a customer-service agent can resolve requests while using tools and following company policies, helping you assess opportunities to reduce manual support work. These results use the Standard airline, retail and telecom evaluation.
updatedHLE-Diamond
What changed · 1 fields
- summary
HLE-Diamond is a revised 1,000-question version of Humanity’s Last Exam, split into 500 reasoning and 500 knowledge questions. These results use the official prompts without tools. Scores are separate from the original Humanity’s Last Exam. → Tests difficult knowledge and reasoning questions, helping you choose a model for demanding analysis before you trust it with client research. This 1,000-question evaluation uses no tools and is separate from the original Humanity's Last Exam.
updatedGDP.pdf
What changed · 1 fields
- summary
GDP.pdf tests reasoning over long professional documents across ten domains. The All-pass score measures attempts satisfying every task criterion. These results use the independent five-attempt evaluation with LiteParse text and page images for models supporting images; other implementations remain separate. → Tests reasoning over long professional documents, helping you assess which model could reduce the time you spend working through source material for client projects. All-pass scores require every criterion to be satisfied; implementations are kept separate.
updatedAA-Briefcase
What changed · 1 fields
- summary
AA-Briefcase evaluates agents producing spreadsheets, presentations, memos and other professional deliverables across 91 tasks. Its combined Elo incorporates rubric performance, analytical quality and presentation. Each task runs independently, and the public Lite example is excluded. → Tests professional deliverables such as spreadsheets, presentations and memos, helping you choose a model for complex client projects with many linked requirements. Rubric scores measure requirements met; Elo also reflects analytical and presentation quality, not hours saved.
updatedIFBench
What changed · 1 fields
- summary
IFBench tests whether models follow precise output requirements and generalize to 58 diverse, verifiable constraint types. It measures instruction-following reliability rather than completion of a full agent workflow. → Tests whether a model follows detailed formatting and output instructions, helping you reduce rework when client deliverables must meet a precise brief. It measures instruction following rather than completion of an entire project.
updatedLiveBench: Agentic
What changed · 1 fields
- summary
LiveBench 2.0 evaluates long-running agentic work on its October 25, 2026 release. This suite contains 18 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests extended tasks that require an agent to keep working toward a goal, helping you assess which model could handle longer assignments with less step-by-step direction. Compare results from the same dated task set.
updatedLiveBench: Design
What changed · 1 fields
- summary
LiveBench 2.0 evaluates document and presentation creation on its October 25, 2026 release. This suite contains 25 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests document and presentation creation, helping you choose a model for preparing client-facing materials with less manual production work. Compare the same dated task set; the score does not measure turnaround time.
updatedLiveBench: Coding
What changed · 1 fields
- summary
LiveBench 2.0 evaluates coding with LiveBench on its October 25, 2026 release. This suite contains 29 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests coding tasks, helping you compare models for implementation work that you would otherwise write or debug yourself. Compare results from the same dated task set and review generated changes before delivery.
updatedLiveBench: Browser Use
What changed · 1 fields
- summary
LiveBench 2.0 evaluates browser workflows on its October 25, 2026 release. This suite contains 25 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests browser workflows, helping you choose an agent for repetitive online tasks so you can spend less time navigating websites yourself. Results from different dated task sets are kept separate.
updatedLiveBench: Data Analysis
What changed · 1 fields
- summary
LiveBench 2.0 evaluates data analysis on its October 25, 2026 release. This suite contains 150 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests data analysis, helping you choose a model to support turning client data into useful findings with less manual processing. Compare the same dated task set; a higher score does not guarantee correct analysis on your data.
updatedLiveBench: Research
What changed · 1 fields
- summary
LiveBench 2.0 evaluates research reports on its October 25, 2026 release. This suite contains 22 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests research report tasks, helping you compare assistants for gathering and organizing material before advising clients. Results are specific to each dated task set and do not measure time saved.
updatedLiveBench: Reasoning
What changed · 1 fields
- summary
LiveBench 2.0 evaluates reasoning with LiveBench on its October 25, 2026 release. This suite contains 202 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests reasoning problems, helping you compare models for working through complex questions and checking your approach. This suite measures reasoning performance rather than completion of a business workflow.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedLiveBench: Agentic
What changed · 1 fields
- summary
LiveBench 2.0 agentic suite on the October 14, 2026 release. Historical results, kept separate from the October 25 release used by Agent Index. Source-specific model effort labels are retained; no effort is inferred. → Tests extended tasks that require an agent to keep working toward a goal, helping you assess which model could handle longer assignments with less step-by-step direction. Compare results from the same dated task set.
updatedLiveBench: Coding
What changed · 1 fields
- summary
LiveBench 2.0 coding suite on the October 14, 2026 release. Historical results, kept separate from the October 25 release used by Agent Index. Source-specific model effort labels are retained; no effort is inferred. → Tests coding tasks, helping you compare models for implementation work that you would otherwise write or debug yourself. Compare results from the same dated task set and review generated changes before delivery.
updatedLiveBench: Browser Use
What changed · 1 fields
- summary
LiveBench 2.0 browser-use suite on the October 14, 2026 release. Historical results, kept separate from the October 25 release used by Agent Index. Source-specific model effort labels are retained; no effort is inferred. → Tests browser workflows, helping you choose an agent for repetitive online tasks so you can spend less time navigating websites yourself. Results from different dated task sets are kept separate.
updatedSWE-bench Verified
What changed · 1 fields
- summary
SWE-bench Verified measures whether an agent can resolve real GitHub issues across 500 human-validated tasks. The primary track uses mini-SWE-agent 2.0.0; provider-reported setups remain separate. → Tests whether an agent can fix real GitHub issues, helping you choose a model for bug fixes and maintenance so you can focus on reviewing changes and serving clients. The Verified set contains 500 human-validated issues; agent setups remain separate.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedDeepFinance
What changed · 1 fields
- summary
DeepFinance tests long-horizon private equity and investment banking work: valuation, due diligence, underwriting and financial modeling. Agents work across data-room files and are graded on deliverables, workflow and judgment. Scores average rubric criteria satisfied across 21 tasks and 10 attempts per task. Native agent harnesses differ by model. → Tests private equity and investment banking work such as valuation, due diligence and financial modeling, helping you assess AI support for demanding client analysis. Scores reflect deliverables, workflow and judgment in model-specific agent setups.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedTerminal-Bench
What changed · 1 fields
- summary
Terminal-Bench evaluates agents on challenging tasks performed through a terminal. It tests multi-step command-line work, problem solving, and the ability to use tools to complete practical tasks. Reported scores depend on the benchmark release and agent harness. → Tests whether an agent can complete multi-step command-line tasks, helping you choose a model for development, debugging and technical operations with less manual execution. Compare the same release and agent setup.
updatedGDP.pdf
What changed · 1 fields
- summary
GDP.pdf tests reasoning over long professional documents across ten domains. The All-pass score measures attempts satisfying every task criterion. These results use the independent five-attempt evaluation with LiteParse text and page images for models supporting images; other implementations remain separate. → Tests reasoning over long professional documents, helping you assess which model could reduce the time you spend working through source material for client projects. All-pass scores require every criterion to be satisfied; implementations are kept separate.
updatedHLE-Diamond
What changed · 1 fields
- summary
HLE-Diamond is a revised 1,000-question version of Humanity’s Last Exam, split into 500 reasoning and 500 knowledge questions. These results use the official prompts without tools. Scores are separate from the original Humanity’s Last Exam. → Tests difficult knowledge and reasoning questions, helping you choose a model for demanding analysis before you trust it with client research. This 1,000-question evaluation uses no tools and is separate from the original Humanity's Last Exam.
updatedMCP-Atlas
What changed · 1 fields
- summary
MCP-Atlas evaluates real-world tool use through the Model Context Protocol. It tests tool discovery, choosing correct arguments, recovering from errors, and combining tool outputs to complete multi-step workflows. The stored results below cover the public subset under the April 2026 protocol. → Tests whether an agent can find and use the right connected tools, helping you assess its ability to complete workflows across your business systems with less manual coordination. These results cover the public April 2026 subset.
updatedAutomationBench
What changed · 1 fields
- summary
AutomationBench tests agents on realistic business workflows across applications and APIs, covering sales, marketing, operations, support, finance, and HR. It evaluates whether agents can coordinate multi-step work and complete tasks. Zapier’s hosted leaderboard measures strict task completion on a private held-out set. → Tests whether an agent can finish business workflows across apps and APIs, helping you assess what sales, support or operations work you could automate to spend more time with clients. The hosted Zapier track measures complete tasks on a private test set.
updatedOmniDocBench
What changed · 1 fields
- summary
OmniDocBench evaluates document parsing across varied layouts, including text, tables and formulas. This result is the publisher's reported v1.5 score. → Tests extracting text, tables and formulas from documents, helping you choose a model for turning client files into usable data with less manual copying. These results use the reported version 1.5 evaluation.
updatedZeroBench
What changed · 1 fields
- summary
ZeroBench evaluates challenging visual reasoning. This result uses the subquestion track, not full-question accuracy. → Tests difficult visual reasoning, helping you assess which model can interpret complex visual material before using it in client analysis. These scores measure the subquestion track, not complete-question accuracy.
updatedVision2Web
What changed · 1 fields
- summary
Vision2Web evaluates agents building functional websites from visual prototypes and requirements. Launch-report scores retain their publisher's setup. → Tests building functional websites from visual prototypes and requirements, helping you compare models for turning a client's design into working code. Scores retain the publisher's evaluation setup.
updatedExploitGym
What changed · 1 fields
- summary
ExploitGym evaluates security agents discovering and exploiting vulnerabilities in controlled software environments. Results here are from OpenAI's GPT-6 Cyber launch evaluation. → Tests finding and exploiting vulnerabilities in controlled software environments, helping authorized security teams assess AI support for testing and remediation work. These launch-reported results do not measure security across an entire client system.
updatedRemoteExploitBench
What changed · 1 fields
- summary
RemoteExploitBench evaluates exploit development against known vulnerabilities in remotely deployed applications in a controlled environment. → Tests exploit development against known vulnerabilities in controlled, remotely deployed apps, helping authorized security teams assess how AI could support validation of security findings. Results do not establish real-world audit coverage.
updatedFrontierCode
What changed · 1 fields
- summary
Evaluates mergeable software changes across 132 repository tasks using functional checks and rubrics. Native agent harnesses and reasoning efforts are recorded per report. → Tests whether an agent can produce software changes that meet functional and review requirements, helping you assess how much implementation work you could delegate before human code review. Scores cover 132 tasks and depend on the agent setup and effort level.
updatedFrontierCode
What changed · 1 fields
- summary
Evaluates mergeable software changes across 132 repository tasks using functional checks and rubrics. Native agent harnesses and reasoning efforts are recorded per report. → Tests whether an agent can produce software changes that meet functional and review requirements, helping you assess how much implementation work you could delegate before human code review. Scores cover 132 tasks and depend on the agent setup and effort level.
updatedFrontierSWE
What changed · 1 fields
- summary
Measures ultra-long-horizon software engineering and research across 34 tasks, with a 20-hour budget per trial and five trials per task. → Tests substantial engineering and research assignments, helping you assess models for complex client work that extends beyond a quick code change. Each of the 34 tasks has a 20-hour trial budget; scores do not translate directly into billable hours saved.
updatedTerminal-Bench-Science
What changed · 1 fields
- summary
Measures scientific research workflows across 70 terminal tasks. These native Claude Code runs use cached resources and offline search; repeated-trial scores remain distinct from other harnesses. → Tests computational research workflows, helping you assess what analysis and execution you could delegate so you can focus on interpreting results. Native-agent and independent setups remain separate, and success does not establish a scientific discovery's validity.
updatedTerminal-Bench-Science
What changed · 1 fields
- summary
Measures scientific research workflows across 70 terminal tasks. These native Claude Code runs use cached resources and offline search; repeated-trial scores remain distinct from other harnesses. → Tests computational research workflows, helping you assess what analysis and execution you could delegate so you can focus on interpreting results. Native-agent and independent setups remain separate, and success does not establish a scientific discovery's validity.
updatedOSWorld
What changed · 1 fields
- summary
OSWorld evaluates computer-use agents on practical tasks in a desktop environment. It tests whether agents can interact with applications and complete multi-step work through a graphical interface. Results below are restricted to the stored 2.1 full-set evaluation. → Tests whether an agent can carry out work through desktop applications, helping you assess which computer tasks you might delegate instead of doing every click yourself. Full-set, offline-subset, partial-credit and strict-success results measure different things.
updatedOSWorld
What changed · 1 fields
- summary
OSWorld evaluates computer-use agents on practical tasks in a desktop environment. It tests whether agents can interact with applications and complete multi-step work through a graphical interface. Results below are restricted to the stored 2.1 full-set evaluation. → Tests whether an agent can carry out work through desktop applications, helping you assess which computer tasks you might delegate instead of doing every click yourself. Full-set, offline-subset, partial-credit and strict-success results measure different things.
updatedAA-Briefcase
What changed · 1 fields
- summary
AA-Briefcase evaluates agents producing spreadsheets, presentations, memos and other professional deliverables across 91 tasks. Its combined Elo incorporates rubric performance, analytical quality and presentation. Each task runs independently, and the public Lite example is excluded. → Tests professional deliverables such as spreadsheets, presentations and memos, helping you choose a model for complex client projects with many linked requirements. Rubric scores measure requirements met; Elo also reflects analytical and presentation quality, not hours saved.
updatedWANDR
What changed · 1 fields
- summary
Evaluates wide web research and structured fact collection across 500 tasks. This modified offline-search evaluation uses soft F1 and differs from Perplexity’s live-web harness. → Tests collecting and organizing facts across a broad research assignment, helping you assess AI support for client research that involves many separate lookups. This offline-search soft-F1 evaluation differs from live-web runs and is not a project-completion percentage.
updatedChartography
What changed · 1 fields
- summary
Tests reading specialized professional charts across 100 tasks. Tool-free and tool-assisted results are recorded separately in the report settings. → Tests reading professional charts accurately, helping you choose a model to support client reports and presentations with less manual interpretation. Results cover 100 tasks; tool-free and tool-assisted settings are recorded separately.
updatedRemote Labor Index
What changed · 1 fields
- summary
Measures completion of real freelance projects against human professional deliverables. Human evaluators judge whether the agent’s work is at least as good as the reference. → Tests whether an agent can complete real freelance projects to a human professional's standard, helping you assess what client work you could delegate to make more time for your clients. The score measures projects completed, not the share of your workload or hours you will save.
updatedSWE-bench Pro
What changed · 1 fields
- summary
Long-horizon repository issues; the system card does not establish a version, so this is not labeled v2 Hard. → Tests substantial software repository issues, helping you assess which model could support complex fixes and implementation work for clients. The source does not establish a specific Pro release, so these scores are not labeled v2 Hard.
updatedSWE-bench Multilingual
What changed · 1 fields
- summary
300 software tasks across nine programming languages. → Tests software fixes across nine programming languages, helping you assess models for supporting client codebases beyond a single language. Results cover 300 tasks and do not measure developer productivity directly.
updatedSWE-bench Multimodal
What changed · 1 fields
- summary
Software issues with screenshots and other visual context. → Tests software issues that include screenshots and visual context, helping you assess which model could connect a client's visible problem to the code changes needed to fix it.
updatedRoboDojo Sim
What changed · 1 fields
- summary
Evaluates agents on simulated robotics tasks. This is Anthropic’s internal run at highest available effort, not an independently reproduced result. → Tests controlling a simulated robot from visual information, helping you assess progress toward automating physical tasks. These are internal simulation results, not evidence of reliable operation in a client's workplace.
updatedCyScenarioBench
What changed · 1 fields
- summary
Measures planning and execution across multi-stage security scenarios. Labs use different private subsets; the Anthropic 10-challenge cohort must not be compared as an identical test to another lab’s score. → Tests multi-stage cybersecurity scenarios, helping authorized security teams assess an agent's ability to plan and execute complex assessments. Labs use different private task sets, and safeguards settings are recorded with each report.
updatedCVE Rediscovery
What changed · 1 fields
- summary
Measures rediscovery of documented security vulnerabilities from vulnerable repository snapshots, without internet access. This cohort contains 1,829 October 2026 advisories. → Tests finding previously documented vulnerabilities from vulnerable code snapshots, helping authorized security teams assess AI support for code audits. This 1,829-advisory evaluation does not establish how many unknown vulnerabilities it would find in a client's system.
updatedDe novo binder design
What changed · 1 fields
- summary
Measures autonomous protein binder design with a 24-hour budget per target, protein-design tools and one H200 GPU. This is a computational evaluation, not evidence of experimentally validated binding. → Tests computational protein-binder design within a fixed time and hardware budget, helping research teams assess AI support for generating candidates. Scores do not establish experimentally validated binding or clinical usefulness.
updatedLiveBench: Agentic
What changed · 1 fields
- summary
LiveBench 2.0 evaluates long-running agentic work on its October 25, 2026 release. This suite contains 18 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests extended tasks that require an agent to keep working toward a goal, helping you assess which model could handle longer assignments with less step-by-step direction. Compare results from the same dated task set.
updatedLiveBench: Coding
What changed · 1 fields
- summary
LiveBench 2.0 evaluates coding with LiveBench on its October 25, 2026 release. This suite contains 29 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests coding tasks, helping you compare models for implementation work that you would otherwise write or debug yourself. Compare results from the same dated task set and review generated changes before delivery.
updatedLiveBench: Browser Use
What changed · 1 fields
- summary
LiveBench 2.0 evaluates browser workflows on its October 25, 2026 release. This suite contains 25 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests browser workflows, helping you choose an agent for repetitive online tasks so you can spend less time navigating websites yourself. Results from different dated task sets are kept separate.
updatedLiveBench: Reasoning
What changed · 1 fields
- summary
LiveBench 2.0 evaluates reasoning with LiveBench on its October 25, 2026 release. This suite contains 202 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests reasoning problems, helping you compare models for working through complex questions and checking your approach. This suite measures reasoning performance rather than completion of a business workflow.
updatedLiveBench: Data Analysis
What changed · 1 fields
- summary
LiveBench 2.0 evaluates data analysis on its October 25, 2026 release. This suite contains 150 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests data analysis, helping you choose a model to support turning client data into useful findings with less manual processing. Compare the same dated task set; a higher score does not guarantee correct analysis on your data.
updatedLiveBench: Research
What changed · 1 fields
- summary
LiveBench 2.0 evaluates research reports on its October 25, 2026 release. This suite contains 22 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests research report tasks, helping you compare assistants for gathering and organizing material before advising clients. Results are specific to each dated task set and do not measure time saved.
updatedLiveBench: Design
What changed · 1 fields
- summary
LiveBench 2.0 evaluates document and presentation creation on its October 25, 2026 release. This suite contains 25 tasks or questions, scored from 0 to 100 in one run per model. Models use their highest supported effort unless a different setting is reported. This is an individual task-suite score, separate from LiveBench’s weighted overall average. → Tests document and presentation creation, helping you choose a model for preparing client-facing materials with less manual production work. Compare the same dated task set; the score does not measure turnaround time.
updatedCyScenarioBench
What changed · 1 fields
- summary
GPT-6 Cyber score on its own task set, kept separate from Anthropic tasks. → Tests multi-stage cybersecurity scenarios, helping authorized security teams assess an agent's ability to plan and execute complex assessments. Labs use different private task sets, and safeguards settings are recorded with each report.
addedCursorBench
Recorded details · 20 fields
- benchmark homepage url
- https://cursor.com/cursorbench
- category
- Coding agents
- created at
- 2026-11-10T01:17:35.861Z
- description
- Ambiguous multi-file coding tasks in Cursor production agent harness; score is not task latency. Current November 9 publisher table includes effort-specific cost, output tokens and steps. Small score differences may be variance.
- display name
- CursorBench
- higher is better
- 1
- id
- cursorbench-4-0
- is primary track
- 1
- is saturated
- Not reported
- is widely used
- Not reported
- maximum score
- 100
- metric
- Score
- minimum score
- 0
- name
- CursorBench
- official url
- https://cursor.com/cursorbench
- page slug
- cursorbench
- summary
- Tests whether an agent can complete ambiguous, multi-file coding tasks from real Cursor sessions, helping you choose a model for client development work with less step-by-step direction. Scores, effort levels and task costs are specific to Cursor's agent setup.
- tracking status
- tracked
- unit
- %
- version
- 4.0 / Cursor agent
addedReal-World Finance
Recorded details · 20 fields
- benchmark homepage url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- category
- Knowledge work
- created at
- 2026-11-10T01:17:35.861Z
- description
- Internal financial analysis evaluation: 123 tasks, one attempt per model, three-hour limit, Opus 5.5 model grader. Grader Elo gaps divided by 2.61 and anchored to preserve the earlier scale. Fable missed six tasks; the headline Elo advantage drops from about 66 to 43 points if counted as losses. Not DeepFinance.
- display name
- Real-World Finance
- higher is better
- 1
- id
- real-world-finance-v2
- is primary track
- 1
- is saturated
- Not reported
- is widely used
- Not reported
- maximum score
- Not reported
- metric
- Elo
- minimum score
- 0
- name
- Real-World Finance
- official url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- page slug
- real-world-finance
- summary
- Compares financial models, audits and other professional finance deliverables, helping you assess AI support for preparing client analysis. Elo reflects a model grader's preferences across 123 tasks; it is not a completion percentage or a measure of time saved.
- tracking status
- tracked
- unit
- Elo
- version
- 2 / November 2026 grader
addedBenchCAD
Recorded details · 20 fields
- benchmark homepage url
- https://benchcad.com/leaderboard
- category
- Knowledge work
- created at
- 2026-11-10T01:17:35.862Z
- description
- 100 Core CAD tasks; five runs; max adaptive thinking; shell, file editor and image zoom; thinking-block clearing; no round or total-token limit; official scorer. Not directly comparable to standard-harness or older-version scores.
- display name
- BenchCAD
- higher is better
- 1
- id
- benchcad-2-1-tools
- is primary track
- 1
- is saturated
- Not reported
- is widely used
- Not reported
- maximum score
- 100
- metric
- Mean partial-credit score
- minimum score
- 0
- name
- BenchCAD
- official url
- https://benchcad.com/leaderboard
- page slug
- benchcad
- summary
- Tests rebuilding parts, assemblies and circuit boards from drawings and images, helping you assess AI support for client design and prototyping work. Scores award partial credit, and tool-assisted runs are separate from standard runs; manufacturing still requires validation.
- tracking status
- tracked
- unit
- %
- version
- 2.1 / Core / with tools
addedBenchCAD
Recorded details · 20 fields
- benchmark homepage url
- https://benchcad.com/leaderboard
- category
- Knowledge work
- created at
- 2026-11-10T01:17:35.862Z
- description
- 100 Core CAD tasks; five runs; max adaptive thinking; Anthropic implementation of standard protocol; render feedback; up to 60 rounds and 500,000 output tokens; official scorer. Without the extra shell/editor/zoom tool setup.
- display name
- BenchCAD
- higher is better
- 1
- id
- benchcad-2-1-standard
- is primary track
- 0
- is saturated
- Not reported
- is widely used
- Not reported
- maximum score
- 100
- metric
- Mean partial-credit score
- minimum score
- 0
- name
- BenchCAD
- official url
- https://benchcad.com/leaderboard
- page slug
- benchcad
- summary
- Tests rebuilding parts, assemblies and circuit boards from drawings and images, helping you assess AI support for client design and prototyping work. Scores award partial credit, and tool-assisted runs are separate from standard runs; manufacturing still requires validation.
- tracking status
- tracked
- unit
- %
- version
- 2.1 / Core / standard
addedHarvey LAB-AA
Recorded details · 20 fields
- benchmark homepage url
- https://artificialanalysis.ai/evaluations/harvey-lab-aa
- category
- Knowledge work
- created at
- 2026-11-10T01:17:35.862Z
- description
- Independent Artificial Analysis Stirrup evaluation on 120 private Harvey tasks; three-judge panel. Every rubric criterion must pass and there must be no material hallucination. Not comparable to v1.0 or ungated all-pass rates.
- display name
- Harvey LAB-AA
- higher is better
- 1
- id
- harvey-lab-aa-v1-1
- is primary track
- 1
- is saturated
- Not reported
- is widely used
- Not reported
- maximum score
- 100
- metric
- Hallucination-gated all-pass rate
- minimum score
- 0
- name
- Harvey LAB-AA
- official url
- https://artificialanalysis.ai/evaluations/harvey-lab-aa
- page slug
- harvey-lab-aa
- summary
- Tests whether an agent can produce legal deliverables that meet every requirement without material hallucinations, helping you assess how much drafting and document work you could delegate before professional review. These results use the private 120-task v1.1 evaluation, not earlier LAB scoring.
- tracking status
- tracked
- unit
- %
- version
- 1.1 / private 120 / hallucination-gated
addedClaude Fable 5.2 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 32.79, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 352794, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 218, "tools": true}
- id
- 1389
- model id
- claude-fable-5-2
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 60
- source location
- CursorBench 4.0 table: Fable 5.2 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 13.43, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 218363, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 185, "tools": true}
- id
- 1390
- model id
- claude-opus-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 57.8
- source location
- CursorBench 4.0 table: Opus 5.5 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 9.7, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 99259, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 69, "tools": true}
- id
- 1391
- model id
- claude-fable-5-2
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 56.9
- source location
- CursorBench 4.0 table: Fable 5.2 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 15.48, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 161251, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 108, "tools": true}
- id
- 1392
- model id
- claude-fable-5-2
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 56.3
- source location
- CursorBench 4.0 table: Fable 5.2 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 6.98, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 101083, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 109, "tools": true}
- id
- 1393
- model id
- claude-opus-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 56
- source location
- CursorBench 4.0 table: Opus 5.5 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 3.97, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 53078, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 68, "tools": true}
- id
- 1394
- model id
- claude-opus-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 56
- source location
- CursorBench 4.0 table: Opus 5.5 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 7.05, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 271920, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 170, "tools": true}
- id
- 1395
- model id
- claude-sonnet-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 55.5
- source location
- CursorBench 4.0 table: Sonnet 5.5 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.66, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 44103, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 35, "tools": true}
- id
- 1396
- model id
- claude-fable-5-2
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 54.8
- source location
- CursorBench 4.0 table: Fable 5.2 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 3.94, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 36616, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 30, "tools": true}
- id
- 1397
- model id
- claude-fable-5-2
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 53.2
- source location
- CursorBench 4.0 table: Fable 5.2 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 2.81, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 100158, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 78, "tools": true}
- id
- 1398
- model id
- claude-sonnet-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 53.1
- source location
- CursorBench 4.0 table: Sonnet 5.5 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGemini 4 Argon · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.57, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 109500, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 118, "tools": true}
- id
- 1399
- model id
- gemini-4-argon
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 52.9
- source location
- CursorBench 4.0 table: Gemini 4 Argon High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGemini 4 Argon · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.39, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 103600, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 115, "tools": true}
- id
- 1400
- model id
- gemini-4-argon
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 52.8
- source location
- CursorBench 4.0 table: Gemini 4 Argon Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 2.91, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 37954, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 54, "tools": true}
- id
- 1401
- model id
- claude-opus-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 52.5
- source location
- CursorBench 4.0 table: Opus 5.5 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 17.28, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 117236, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 128, "tools": true}
- id
- 1402
- model id
- claude-fable-5-1
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 51.8
- source location
- CursorBench 4.0 table: Fable 5.1 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 13.01, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 87294, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 101, "tools": true}
- id
- 1403
- model id
- claude-fable-5-1
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 51.6
- source location
- CursorBench 4.0 table: Fable 5.1 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGemini 4 Argon · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.09, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 94200, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 110, "tools": true}
- id
- 1404
- model id
- gemini-4-argon
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 50.6
- source location
- CursorBench 4.0 table: Gemini 4 Argon Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 9.08, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 58438, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 77, "tools": true}
- id
- 1405
- model id
- claude-fable-5-1
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 49.2
- source location
- CursorBench 4.0 table: Fable 5.1 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Haiku 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.12, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 325934, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 163, "tools": true}
- id
- 1406
- model id
- claude-haiku-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 48.4
- source location
- CursorBench 4.0 table: Haiku 5.5 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.2, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 37391, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 41, "tools": true}
- id
- 1407
- model id
- claude-sonnet-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 47.8
- source location
- CursorBench 4.0 table: Sonnet 5.5 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedDeepSeek V4.1 Flash 1017 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.98, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 537047, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 187, "tools": true}
- id
- 1408
- model id
- deepseek-v4-1-flash-1017
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 47.7
- source location
- CursorBench 4.0 table: DeepSeek V4.1 Flash 1017 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 7.05, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 45411, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 63, "tools": true}
- id
- 1409
- model id
- claude-fable-5-1
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 46.8
- source location
- CursorBench 4.0 table: Fable 5.1 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 11.95, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 85384, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 106, "tools": true}
- id
- 1410
- model id
- claude-opus-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 46.6
- source location
- CursorBench 4.0 table: Opus 5 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGrok 4.7 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 6.01, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 70141, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 88, "tools": true}
- id
- 1411
- model id
- grok-4-7
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 46.3
- source location
- CursorBench 4.0 table: Grok 4.7 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedDeepSeek V4.1 Flash 1017 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.8, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 398255, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 189, "tools": true}
- id
- 1412
- model id
- deepseek-v4-1-flash-1017
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 46.3
- source location
- CursorBench 4.0 table: DeepSeek V4.1 Flash 1017 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 11.43, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 80094, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 103, "tools": true}
- id
- 1413
- model id
- claude-opus-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 46.1
- source location
- CursorBench 4.0 table: Opus 5 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedDeepSeek V4.1 Flash 1017 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.77, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 385196, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 186, "tools": true}
- id
- 1414
- model id
- deepseek-v4-1-flash-1017
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 45.6
- source location
- CursorBench 4.0 table: DeepSeek V4.1 Flash 1017 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 5.44, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 34795, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 51, "tools": true}
- id
- 1415
- model id
- claude-fable-5-1
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 45.1
- source location
- CursorBench 4.0 table: Fable 5.1 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedDeepSeek V4.1 Flash 1017 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.6, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 290651, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 161, "tools": true}
- id
- 1416
- model id
- deepseek-v4-1-flash-1017
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 44.9
- source location
- CursorBench 4.0 table: DeepSeek V4.1 Flash 1017 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 9.0, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 61405, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 86, "tools": true}
- id
- 1417
- model id
- claude-opus-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 44.7
- source location
- CursorBench 4.0 table: Opus 5 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Haiku 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.56, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 143813, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 93, "tools": true}
- id
- 1418
- model id
- claude-haiku-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 44.3
- source location
- CursorBench 4.0 table: Haiku 5.5 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGrok 4.7 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.69, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 56382, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 71, "tools": true}
- id
- 1419
- model id
- grok-4-7
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 43.9
- source location
- CursorBench 4.0 table: Grok 4.7 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.17, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 15811, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 28, "tools": true}
- id
- 1420
- model id
- claude-opus-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 43.7
- source location
- CursorBench 4.0 table: Opus 5.5 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 6.94, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 45272, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 72, "tools": true}
- id
- 1421
- model id
- claude-opus-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 43.3
- source location
- CursorBench 4.0 table: Opus 5 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGLM-5.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.862Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 5.05, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 96387, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 166, "tools": true}
- id
- 1422
- model id
- glm-5-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 42.6
- source location
- CursorBench 4.0 table: GLM 5.3 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Haiku 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.32, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 77057, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 59, "tools": true}
- id
- 1423
- model id
- claude-haiku-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 42.3
- source location
- CursorBench 4.0 table: Haiku 5.5 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Sol · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 8.23, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 42944, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 99, "tools": true}
- id
- 1424
- model id
- gpt-5-6-sol
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 41.7
- source location
- CursorBench 4.0 table: GPT-5.6 Sol Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGrok 4.7 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 3.49, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 36683, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 60, "tools": true}
- id
- 1425
- model id
- grok-4-7
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 41.6
- source location
- CursorBench 4.0 table: Grok 4.7 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedMuse Spark 1.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 2.64, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 52005, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 98, "tools": true}
- id
- 1426
- model id
- muse-spark-1-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 41.6
- source location
- CursorBench 4.0 table: Muse Spark 1.3 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Terra · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 5.14, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 60814, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 107, "tools": true}
- id
- 1427
- model id
- gpt-5-6-terra
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 41.3
- source location
- CursorBench 4.0 table: GPT-5.6 Terra Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.87, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 31995, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 57, "tools": true}
- id
- 1428
- model id
- claude-opus-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 40.7
- source location
- CursorBench 4.0 table: Opus 5 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGemini 3.8 Flash · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.7, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 162565, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 324, "tools": true}
- id
- 1429
- model id
- gemini-3-8-flash
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 39.6
- source location
- CursorBench 4.0 table: Gemini 3.8 Flash High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.52, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 16036, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 22, "tools": true}
- id
- 1430
- model id
- claude-sonnet-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 39.2
- source location
- CursorBench 4.0 table: Sonnet 5.5 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGLM-5.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 3.24, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 60031, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 114, "tools": true}
- id
- 1431
- model id
- glm-5-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 38
- source location
- CursorBench 4.0 table: GLM 5.3 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Sol · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.4, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 24729, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 55, "tools": true}
- id
- 1432
- model id
- gpt-5-6-sol
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 37.7
- source location
- CursorBench 4.0 table: GPT-5.6 Sol Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedMuse Spark 1.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 2.1, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 40891, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 83, "tools": true}
- id
- 1433
- model id
- muse-spark-1-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 37.5
- source location
- CursorBench 4.0 table: Muse Spark 1.3 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGemini 3.8 Flash · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.06, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 128364, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 290, "tools": true}
- id
- 1434
- model id
- gemini-3-8-flash
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 37.3
- source location
- CursorBench 4.0 table: Gemini 3.8 Flash Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Haiku 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.17, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 42659, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 39, "tools": true}
- id
- 1435
- model id
- claude-haiku-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 36.9
- source location
- CursorBench 4.0 table: Haiku 5.5 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGLM-5.3-Flash · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.39, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 56410, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 118, "tools": true}
- id
- 1436
- model id
- glm-5-3-flash
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 36.8
- source location
- CursorBench 4.0 table: GLM 5.3 Flash Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Luna · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.03, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 87284, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 208, "tools": true}
- id
- 1437
- model id
- gpt-5-6-luna
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 35.9
- source location
- CursorBench 4.0 table: GPT-5.6 Luna Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.37, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 11668, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 18, "tools": true}
- id
- 1438
- model id
- claude-sonnet-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 35.8
- source location
- CursorBench 4.0 table: Sonnet 5.5 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Sol · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 2.85, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 16174, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 41, "tools": true}
- id
- 1439
- model id
- gpt-5-6-sol
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 35.7
- source location
- CursorBench 4.0 table: GPT-5.6 Sol High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 7.17, "effort": "max", "harness": "Cursor production agent", "output_tokens_per_task": 149257, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 140, "tools": true}
- id
- 1440
- model id
- claude-sonnet-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 34.1
- source location
- CursorBench 4.0 table: Sonnet 5 Max
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Terra · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.81, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 23436, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 43, "tools": true}
- id
- 1441
- model id
- gpt-5-6-terra
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 33.6
- source location
- CursorBench 4.0 table: GPT-5.6 Terra Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedMuse Spark 1.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.66, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 30654, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 69, "tools": true}
- id
- 1442
- model id
- muse-spark-1-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 33.4
- source location
- CursorBench 4.0 table: Muse Spark 1.3 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGLM-5.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 2.04, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 31983, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 81, "tools": true}
- id
- 1443
- model id
- glm-5-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 33.3
- source location
- CursorBench 4.0 table: GLM 5.3 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGrok 4.7 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.58, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 15677, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 40, "tools": true}
- id
- 1444
- model id
- grok-4-7
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 33.1
- source location
- CursorBench 4.0 table: Grok 4.7 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Luna · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.44, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 40598, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 98, "tools": true}
- id
- 1445
- model id
- gpt-5-6-luna
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 33
- source location
- CursorBench 4.0 table: GPT-5.6 Luna Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedMuse Spark 1.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.49, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 27255, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 64, "tools": true}
- id
- 1446
- model id
- muse-spark-1-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 32.6
- source location
- CursorBench 4.0 table: Muse Spark 1.3 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 4.55, "effort": "xhigh", "harness": "Cursor production agent", "output_tokens_per_task": 83373, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 102, "tools": true}
- id
- 1447
- model id
- claude-sonnet-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 32
- source location
- CursorBench 4.0 table: Sonnet 5 Extra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Sol · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.77, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 10111, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 32, "tools": true}
- id
- 1448
- model id
- gpt-5-6-sol
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 31.1
- source location
- CursorBench 4.0 table: GPT-5.6 Sol Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGLM-5.3-Flash · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.25, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 35104, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 84, "tools": true}
- id
- 1449
- model id
- glm-5-3-flash
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 31.1
- source location
- CursorBench 4.0 table: GLM 5.3 Flash High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Haiku 5.5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.08, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 23382, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 25, "tools": true}
- id
- 1450
- model id
- claude-haiku-5-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 30.9
- source location
- CursorBench 4.0 table: Haiku 5.5 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 3.48, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 61146, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 85, "tools": true}
- id
- 1451
- model id
- claude-sonnet-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 30.8
- source location
- CursorBench 4.0 table: Sonnet 5 High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Terra · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.11, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 13162, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 33, "tools": true}
- id
- 1452
- model id
- gpt-5-6-terra
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 30.7
- source location
- CursorBench 4.0 table: GPT-5.6 Terra High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Luna · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.25, "effort": "high", "harness": "Cursor production agent", "output_tokens_per_task": 23368, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 64, "tools": true}
- id
- 1453
- model id
- gpt-5-6-luna
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 29.4
- source location
- CursorBench 4.0 table: GPT-5.6 Luna High
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedMuse Spark 1.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.93, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 17483, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 47, "tools": true}
- id
- 1454
- model id
- muse-spark-1-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 29.3
- source location
- CursorBench 4.0 table: Muse Spark 1.3 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 2.31, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 39114, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 65, "tools": true}
- id
- 1455
- model id
- claude-sonnet-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 28
- source location
- CursorBench 4.0 table: Sonnet 5 Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Terra · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.64, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 7307, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 25, "tools": true}
- id
- 1456
- model id
- gpt-5-6-terra
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 27.6
- source location
- CursorBench 4.0 table: GPT-5.6 Terra Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGLM-5.3-Flash · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.15, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 17831, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 58, "tools": true}
- id
- 1457
- model id
- glm-5-3-flash
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 26.9
- source location
- CursorBench 4.0 table: GLM 5.3 Flash Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Terra · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.52, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 5914, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 23, "tools": true}
- id
- 1458
- model id
- gpt-5-6-terra
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 25.2
- source location
- CursorBench 4.0 table: GPT-5.6 Terra Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Sol · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.87, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 4885, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 21, "tools": true}
- id
- 1459
- model id
- gpt-5-6-sol
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 24.6
- source location
- CursorBench 4.0 table: GPT-5.6 Sol Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedMuse Spark 1.3 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.56, "effort": "minimal", "harness": "Cursor production agent", "output_tokens_per_task": 10620, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 34, "tools": true}
- id
- 1460
- model id
- muse-spark-1-3
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 24.3
- source location
- CursorBench 4.0 table: Muse Spark 1.3 Minimal
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Sonnet 5 · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 1.39, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 23772, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 46, "tools": true}
- id
- 1461
- model id
- claude-sonnet-5
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 24.1
- source location
- CursorBench 4.0 table: Sonnet 5 Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Luna · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.08, "effort": "medium", "harness": "Cursor production agent", "output_tokens_per_task": 7642, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 32, "tools": true}
- id
- 1462
- model id
- gpt-5-6-luna
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 22.2
- source location
- CursorBench 4.0 table: GPT-5.6 Luna Medium
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedGPT-5.6 Luna · CursorBench
Recorded details · 15 fields
- benchmark id
- cursorbench-4-0
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "cursorbench-4-0", "cost_usd_per_task": 0.03, "effort": "low", "harness": "Cursor production agent", "output_tokens_per_task": 3288, "settings": "Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant.", "steps_per_task": 18, "tools": true}
- id
- 1463
- model id
- gpt-5-6-luna
- notes
- {"attribution": "Cursor", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- Not reported
- score
- 16
- source location
- CursorBench 4.0 table: GPT-5.6 Luna Low
- source publisher
- Cursor
- source title
- CursorBench 4.0 publisher leaderboard
- source type
- benchmark_publisher
- source url
- https://cursor.com/cursorbench
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · Real-World Finance
Recorded details · 15 fields
- benchmark id
- real-world-finance-v2
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "real-world-finance-v2", "effort": "max", "settings": "123 tasks; one attempt; three-hour deadline; Opus 5.5 grader; rescaled Elo. Fable missed six tasks; Opus completed all 123. Elo is not completion percentage.", "tools": true}
- id
- 1464
- model id
- claude-fable-5-2
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 1637
- source location
- Section 8.14.6, PDF pages 241-242
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · Real-World Finance
Recorded details · 15 fields
- benchmark id
- real-world-finance-v2
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "real-world-finance-v2", "effort": "max", "settings": "123 tasks; one attempt; three-hour deadline; Opus 5.5 grader; rescaled Elo. Fable missed six tasks; Opus completed all 123. Elo is not completion percentage.", "tools": true}
- id
- 1465
- model id
- claude-opus-5-5
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 1571
- source location
- Section 8.14.6, PDF pages 241-242
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · Real-World Finance
Recorded details · 15 fields
- benchmark id
- real-world-finance-v2
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "real-world-finance-v2", "effort": "max", "settings": "123 tasks; one attempt; three-hour deadline; Opus 5.5 grader; rescaled Elo. Fable missed six tasks; Opus completed all 123. Elo is not completion percentage.", "tools": true}
- id
- 1466
- model id
- claude-fable-5-1
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 1493
- source location
- Section 8.14.6, PDF pages 241-242
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · BenchCAD
Recorded details · 15 fields
- benchmark id
- benchcad-2-1-standard
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "benchcad-2-1-standard", "effort": "max", "settings": "100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart.", "tools": false}
- id
- 1467
- model id
- claude-fable-5-2
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 75.4
- source location
- Section 8.13.2, PDF pages 232-233, Figure 8.13.2.A
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · BenchCAD
Recorded details · 15 fields
- benchmark id
- benchcad-2-1-tools
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "benchcad-2-1-tools", "effort": "max", "settings": "100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart.", "tools": true}
- id
- 1468
- model id
- claude-fable-5-2
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 83.5
- source location
- Section 8.13.2, PDF pages 232-233, Figure 8.13.2.A
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · BenchCAD
Recorded details · 15 fields
- benchmark id
- benchcad-2-1-standard
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "benchcad-2-1-standard", "effort": "max", "settings": "100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart.", "tools": false}
- id
- 1469
- model id
- claude-opus-5-5
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 49
- source location
- Section 8.13.2, PDF pages 232-233, Figure 8.13.2.A
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · BenchCAD
Recorded details · 15 fields
- benchmark id
- benchcad-2-1-tools
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "benchcad-2-1-tools", "effort": "max", "settings": "100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart.", "tools": true}
- id
- 1470
- model id
- claude-opus-5-5
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 65.3
- source location
- Section 8.13.2, PDF pages 232-233, Figure 8.13.2.A
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · BenchCAD
Recorded details · 15 fields
- benchmark id
- benchcad-2-1-standard
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "benchcad-2-1-standard", "effort": "max", "settings": "100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart.", "tools": false}
- id
- 1471
- model id
- claude-fable-5-1
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 36.3
- source location
- Section 8.13.2, PDF pages 232-233, Figure 8.13.2.A
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · BenchCAD
Recorded details · 15 fields
- benchmark id
- benchcad-2-1-tools
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "benchcad-2-1-tools", "effort": "max", "settings": "100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart.", "tools": true}
- id
- 1472
- model id
- claude-fable-5-1
- notes
- {"attribution": "Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 44.4
- source location
- Section 8.13.2, PDF pages 232-233, Figure 8.13.2.A
- source publisher
- Anthropic
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- provider_reported
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.2 · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "max", "harness": "Stirrup", "settings": "120 private tasks; v1.1 three-judge hallucination-gated all-pass rate. Distinct from ungated 10.8% Fable all-pass and earlier v1.0.", "tools": true}
- id
- 1473
- model id
- claude-fable-5-2
- notes
- {"attribution": "Independent results quoted by Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 7.2
- source location
- Section 8.14.2, PDF page 238
- source publisher
- Artificial Analysis via Anthropic system card
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- independent
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Fable 5.1 · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "max", "harness": "Stirrup", "settings": "120 private tasks; v1.1 three-judge hallucination-gated all-pass rate. Distinct from ungated 10.8% Fable all-pass and earlier v1.0.", "tools": true}
- id
- 1474
- model id
- claude-fable-5-1
- notes
- {"attribution": "Independent results quoted by Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 6.4
- source location
- Section 8.14.2, PDF page 238
- source publisher
- Artificial Analysis via Anthropic system card
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- independent
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedClaude Opus 5.5 · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "max", "harness": "Stirrup", "settings": "120 private tasks; v1.1 three-judge hallucination-gated all-pass rate. Distinct from ungated 10.8% Fable all-pass and earlier v1.0.", "tools": true}
- id
- 1475
- model id
- claude-opus-5-5
- notes
- {"attribution": "Independent results quoted by Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 4.2
- source location
- Section 8.14.2, PDF page 238
- source publisher
- Artificial Analysis via Anthropic system card
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- independent
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedGPT-6 Astra · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "max", "harness": "Stirrup", "settings": "120 private tasks; v1.1 three-judge hallucination-gated all-pass rate. Distinct from ungated 10.8% Fable all-pass and earlier v1.0.", "tools": true}
- id
- 1476
- model id
- gpt-6-astra
- notes
- {"attribution": "Independent results quoted by Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 8.6
- source location
- Section 8.14.2, PDF page 238
- source publisher
- Artificial Analysis via Anthropic system card
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- independent
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedGPT-6.1 Sol · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "max", "harness": "Stirrup", "settings": "120 private tasks; v1.1 three-judge hallucination-gated all-pass rate. Distinct from ungated 10.8% Fable all-pass and earlier v1.0.", "tools": true}
- id
- 1477
- model id
- gpt-6-1-sol
- notes
- {"attribution": "Independent results quoted by Anthropic", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-11-09
- score
- 6.9
- source location
- Section 8.14.2, PDF page 238
- source publisher
- Artificial Analysis via Anthropic system card
- source title
- Claude Fable 5.2 and Mythos 5.2 System Card
- source type
- independent
- source url
- https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf
- status
- published
- supersedes result id
- Not reported
addedGrok 4.7 · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "xhigh", "harness": "Stirrup", "settings": "October 8 published v1.1 snapshot; hallucination-gated all-pass rate; not an inferred latest leaderboard value.", "tools": true}
- id
- 1478
- model id
- grok-4-7
- notes
- {"attribution": "Artificial Analysis", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-10-08
- score
- 9.4
- source location
- Score section: October 8 snapshot
- source publisher
- Artificial Analysis
- source title
- Harvey LAB-AA v1.1 launch results
- source type
- independent
- source url
- https://artificialanalysis.ai/articles/harvey-lab-aa-v1-1
- status
- published
- supersedes result id
- Not reported
addedMuse Spark 1.3 · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "max", "harness": "Stirrup", "settings": "October 8 published v1.1 snapshot; hallucination-gated all-pass rate; not an inferred latest leaderboard value.", "tools": true}
- id
- 1479
- model id
- muse-spark-1-3
- notes
- {"attribution": "Artificial Analysis", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-10-08
- score
- 8.9
- source location
- Score section: October 8 snapshot
- source publisher
- Artificial Analysis
- source title
- Harvey LAB-AA v1.1 launch results
- source type
- independent
- source url
- https://artificialanalysis.ai/articles/harvey-lab-aa-v1-1
- status
- published
- supersedes result id
- Not reported
addedKimi K3 · Harvey LAB-AA
Recorded details · 15 fields
- benchmark id
- harvey-lab-aa-v1-1
- created at
- 2026-11-10T01:17:35.863Z
- evaluation settings
- {"comparison_group": "harvey-lab-aa-v1-1", "effort": "max", "harness": "Stirrup", "settings": "October 8 published v1.1 snapshot; hallucination-gated all-pass rate; not an inferred latest leaderboard value.", "tools": true}
- id
- 1480
- model id
- kimi-k3
- notes
- {"attribution": "Artificial Analysis", "checked_at": "2026-11-09", "source_snapshot": "database/benchmarks/additions-2026-11-09"}
- reported at
- 2026-10-08
- score
- 5.3
- source location
- Score section: October 8 snapshot
- source publisher
- Artificial Analysis
- source title
- Harvey LAB-AA v1.1 launch results
- source type
- independent
- source url
- https://artificialanalysis.ai/articles/harvey-lab-aa-v1-1
- status
- published
- supersedes result id
- Not reported
