While SWE-bench Verified is not a perfect benchmark for coding, AFAIK, this is t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		primaprashant 1 day ago \| parent \| context \| favorite \| on: DeepSeek v4 While SWE-bench Verified is not a perfect benchmark for coding, AFAIK, this is the first open-weights model that has crossed the threshold of 80% score on this by scoring 80.6%. Back in Nov 2025, Opus 4.5 (80.9%) was the first proprietary model to do so.

		help

stared 1 day ago [–]

SWE-bench Verified is, at this point, contaminated https://openai.com/index/why-we-no-longer-evaluate-swe-bench...

So it os hard to tell how much of a model gain is due to skill, and how much - overfitting.

Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact