[tag]
8 stories
The new model scores 65 on the Artificial Analysis Intelligence Index but costs $26.54 per task and takes 44.6 minutes to complete, making it a premium option for complex, high-stakes workloads.
Three new papers from arXiv evaluate how AI agents handle operational state, long-term memory, and real-world collaborative workflows, revealing significant gaps in current model performance.
A new benchmark using Claude Opus 5.5 reveals that framework choice significantly impacts the token cost and time required for AI agents to build web applications.
The new frontier model claims a 10x improvement in patch generation and perfect prompt-injection resistance, but remains locked behind a staged release and temporary API pricing.
A new framework applies Rasch measurement theory to benchmark large language models, offering a more rigorous way to compare model proficiency and item difficulty.
Researchers show that AI models fine-tuned on popular benchmarks inflate their scores by memorizing, not learning, and propose a framework to measure the real-world performance gap.
A new open-source tool turns completed PRs and coding sessions into hidden-test tasks to compare AI coding agents on your codebase.
A research paper introduces a suite of coding agent benchmarks designed to evaluate progress on the ARC-AGI abstraction and reasoning corpus.
