NIST says DeepSeek V4 Pro trails the frontier by about eight months

Start here

In everyday words

CAISI ran a set of tests across different skills and summarized where DeepSeek V4 Pro sits compared with other leading models and earlier generations.

Need a meaning?

Quick Sip

What you need to know

Who is affected: researchers, technical leaders, AI-watchers
What changed: On May 1, 2026, NIST’s CAISI published results from its evaluation of the open-weight model DeepSeek V4 Pro, reporting that it lags the frontier by about eight months across a multi-domain benchmark suite.
Why it matters: Independent evaluations can reduce hype and make cross-model comparisons more reliable. They also help policymakers and buyers understand what “open-weight” systems can and cannot do in sensitive areas like cyber and coding.
What to watch next: Watch for follow-up disclosures on CAISI’s non-public benchmarks and whether other labs publish comparable, method-forward evaluations for open-weight releases.

Four useful details

CAISI calls DeepSeek V4 Pro the most capable PRC model it has evaluated so far.
Reported capability lag is based on benchmarks across five domains, including cyber and software engineering.
The report contrasts CAISI results with the developer’s self-reported evaluations.

arXiv · Research PaperPosition: Behavioral Systems Require Behavioral Tests

Adds source-backed context on ai research from arXiv.