AI Builders Digest
Bilingual edition · 双语对照版
第 22 期|2026-06-09|双语精选版|8 条精选|8 位作者|4 个主题 返回目录
编者导语 / Editor's Note

今天最大的一条来自 Swyx:METR 发布 FrontierCode benchmark,发现 SWEBench 超过一半的结果是不可合并的 slop。Opus 4.8 在 FC Diamond 上只得了 13.8%——说明真正的生产级代码质量比通过测试难得多。他把 AI coding benchmark 分成三个时代:2021 的 autocomplete(HumanEval)、2023 的通过测试(SWEBench)、2026 的可维护代码(FrontierCode)。Sam Altman 发布了 OpenAI 的当前计划,5834 赞。Josh Woodward 宣布 NotebookLM 现在可以搜索自有源文件之外的内容,还能输出 PDF/DOCX/XLSX/PPTX。Ben Cherny 发布了 Claude Code 一周年的内部故事。Levie 说不管模型多智能,context 的需求不会消失。

Theme 01

FrontierCode: The Third Era of AI Coding Benchmarks / FrontierCode:AI 编码基准测试的第三个时代

METR 的研究揭示了当前 benchmark 的根本问题,并提出了一个新的、更严格的标准。

Swyx avatarS
Swyx
Writer / Builder
@swyx
中文

METR 的 FrontierCode benchmark 发现 SWEBench 超过一半的结果是不可合并的 slop。FrontierCode 代表 1000 多小时维护者验证的软件工程工作,有 3000+ 评分标准覆盖代码质量和反作弊/奖励黑客。Opus 4.8 在 FC Diamond 上只有 13.8%——前沿模型连解都很难,更别说高质量地解。

Swyx 把 AI coding benchmark 分成三个时代:2021 autocomplete(HumanEval)、2023 通过测试(SWEBench、TerminalBench)、2026 可维护代码(FrontierCode)。

关键图表:FC 最简单的三分之一在 2025 年底被迅速解决——Opus 4 个月从 41% 几乎翻倍到 74%。这描述了 DHH 到 Karpathy 都提到的「2025 年 12 月到底发生了什么」的 vibe shift。每个 FC 层级将依次饱和,大概每年一个。631 赞。

METR 发现 SWEBench 一半是 slop。FrontierCode 1000+ 小时维护者验证、3000+ 评分标准。Opus 4.8 在 FC Diamond 只有 13.8%。

三个时代:Autocomplete→通过测试→可维护代码。2025 底 vibe shift 让 Opus 4 月翻倍。631 赞。

English

METR's FrontierCode benchmark finds that more than half of SWEBench results is unmergeable slop. FrontierCode represents 1000+ hours of maintainer-validated software engineering work with 3000+ rubrics covering code quality and anticheat/reward hacking. Opus 4.8 scores 13.8% on FC Diamond—so hard that frontier models can't yet solve, much less solve with high quality.

Swyx frames three eras of AI coding benchmarks: 2021 autocomplete (HumanEval), 2023 passing tests (SWEBench, TerminalBench), 2026 maintainable code (FrontierCode).

The key chart: the easiest third of FC tasks were rapidly solved over late 2025—Opus almost doubled from 41% to 74% in 4 months. This describes the 'WTF happened in Dec 2025' vibe shift that DHH to Karpathy have called out. Each FC tier will saturate in sequence, hopefully ~annually. 631 likes.

METR found that more than half of SWEBench results is unmergeable slop. FrontierCode represents over 1000+ hours of maintainer validated software engineering work most frontier models cannot yet solve.

Three eras of AI coding benchmarks: 2021 Autocomplete (HumanEval), 2023 Passing Tests (SWEBench, TerminalBench), 2026 Maintainable Code (FrontierCode).

The easiest third of FC tasks were rapidly solved over late 2025 - Opus almost doubled from 41% to 74% in 4 months. This describes the 'WTF happened in Dec 2025' vibe shift.

Theme 02

OpenAI's Plan & NotebookLM Upgrades / OpenAI 路线图与 NotebookLM 升级

Sam Altman 发布 OpenAI 当前计划、NotebookLM 扩展到外部搜索和新输出格式。

Sam Altman avatarSA
Sam Altman
CEO @ OpenAI
@sama
中文

Sam Altman 发布了 OpenAI 的当前计划。5834 赞——今天第二高的互动量,明确信号是市场高度关注 OpenAI 的路线图方向。

Altman 发布 OpenAI 当前计划。5834 赞。

English

Sam Altman shares OpenAI's current plan. 5834 likes—this is the second highest engagement of the day and a clear signal that the market is paying close attention to OpenAI's roadmap direction.

Here is our current plan for OpenAI.

Josh Woodward avatarJW
Josh Woodward
VP @ Google Labs
@joshwoodward
中文

Josh Woodward 宣布 NotebookLM 的杀手级新功能:轻松把搜索扩展到自有源文件之外。加上新的输出格式:PDF、DOCX、XLSX、PPTX、图表。745 赞。

搜索扩展很重要——把 NotebookLM 从封闭源研究工具变成了可以拉取 web 同时锚定自己文档的开放研究平台。

NotebookLM 可搜索自有源文件之外的内容,新输出格式 PDF/DOCX/XLSX/PPTX。745 赞。

English

Josh Woodward announces NotebookLM's killer new feature: easily expand your search beyond your own source files. Plus new output formats: PDFs, DOCX, XLSX, PPTX, charts. 'We want NotebookLM to keep helping you do better research.' 745 likes.

The search expansion is significant—it turns NotebookLM from a closed-source research tool into an open research platform that can pull from the web while grounding in your own documents.

The new killer NotebookLM feature: easily being able to expand your search beyond your own source files. Then, with today's update, you can also make new output formats: PDFs, DOCX, XLSX, PPTX, charts, etc.

Theme 03

Claude Code One Year Later & Context Forever / Claude Code 一周年与 Context 永恒

Ben Cherny 发内部故事、Levie 论证 context 的需求不会消失、Peter Yang 发现两个世界的差距。

Author avatar
中文

Ben Cherny 分享了 Claude Code 一周年的故事:当初内部 demo 时 Slack 上只有两种反应。GA 一年后,他和 Cat Wu 聊了为什么现在用 auto mode 不用 plan mode、routines 怎么在他看到之前就修 bug、为什么他现在大部分编码在手机上做、以及产品走向。1794 赞。

Cherny Claude Code 一周年:内部 demo 两种反应。auto mode、routines 自动修 bug、手机编码。1794 赞。

English

Ben Cherny shares the Claude Code one-year story: when they first demoed it internally, it got two reactions on Slack. A year after GA, he and Cat Wu discuss why he uses auto mode instead of plan mode, how routines fix bugs before he sees them, why he does most of his coding from his phone now, and where the product is going. 1794 likes.

When we first demoed Claude Code internally, it got two reactions on Slack. A year after GA, @_catwu and I sat down to talk about what's changed: why I use auto mode instead of plan mode, how routines fix bugs before I see them, why I do most of my coding from my phone now, and where the product is going.

Aaron Levie avatarAL
Aaron Levie
CEO @ Box
@levie
中文

Levie 论证为什么 context 永远重要:不管模型多智能,都不能替代 context 的需求。任何足够通用的 AI 都需要你引导它——律师、工程师、金融分析师、医疗专业人员用的是同一个模型,但方向完全不同。

指令、领域上下文和专有数据永远需要进入 context window。这就是为什么 AI 自动化不是免费的,也是为什么谁在从 AI 获得收益、谁没有的差距如此之大。你必须投入真正的工作,才能在另一端获得真正的价值。216 赞。

Levie:不管模型多智能 context 永远需要。指令+领域上下文+专有数据进 context window。投入真正工作才能获得价值。216 赞。

English

Levie's thesis on why context will always matter: there's no amount of intelligence that replaces the need for context. For any sufficiently general-purpose AI, you will always have to guide it—lawyers, engineers, financial analysts, healthcare professionals all use the same model but need completely different directions.

Instructions, domain context, and proprietary data will always need to get into the context window. This is why AI automation doesn't come for free, and why there's a wide spectrum of who's getting gains and who's not. You have to put in real work, and you get real value on the other end. 216 likes.

There's no amount of intelligence that can get packed into AI models that replaces the need for context. For any sufficiently general purpose AI, you will always have to guide it in the direction you want as it has an infinite range of directions it can go in.

Instructions, domain context, and proprietary data will always need to get into the context window for the model to be useful. You have to put in real work, and you get real value on the other end.

Peter Yang avatarPY
Peter Yang
Product @ Roblox
@petergyang
中文

Peter Yang 指出一个正在扩大的差距:$200/月补贴订阅上的 AI builder 和试图控制 API 成本的企业员工,面临着完全不同的最佳实践。325 赞。

消费者补贴实验和企业成本意识之间的差距,将越来越决定谁能获得 AI 杠杆、谁不能。

Peter Yang:$200 补贴订阅和企业控成本的两套最佳实践完全不同。325 赞。

English

Peter Yang identifies a growing divide: there's a completely different set of best practices for AI builders on $200/month subsidized subscriptions vs employees at companies trying not to overspend API costs. 325 likes.

This gap between consumer-subsidized experimentation and enterprise cost-consciousness will increasingly define who gets AI leverage and who doesn't.

Feels like there's a completely different set of best practices for AI builders on the $200 / month subsidized subscriptions vs employees working at companies that are trying not to overspend API costs.

Theme 04

The $200/Month Divide & VC Reality / $200 月费鸿沟与 VC 现实

Nikunj 的 VC 讽刺、Amanda Askell 的 Claude 梗、Peter Yang 找 Google 的 coding agent 等价物。

Nikunj Kothari avatarNK
Nikunj Kothari
Partner @ FPV Ventures
@nikunj
中文

Nikunj 的 VC 现实核查:「最搞笑的短信来自那些见了 thesis-driven GP 的创始人,以为 GP 能完全理解他们在做什么……然后发现 thesis 是 associate 写的(更糟的是实习生写的)。」别总信 VC 在网上写的。52 赞。

Nikunj:创始人和 thesis-driven GP 见面后发现 thesis 是 associate 或实习生写的。别总信 VC 网上写的。52 赞。

English

Nikunj's VC reality check: 'The funniest texts are from founders who meet thesis-driven GPs hoping they'll understand EXACTLY what they are building... and then realize the thesis was written and built by an associate (or worse, an intern).' Don't always read what the VC writes on the internet. 52 likes.

The funniest texts are from founders who meet 'thesis driven' GPs hoping they'll understand EXACTLY what they are building.. And then realize the thesis was written and built by an associate (or, worse an intern). Don't always read what the VC writes on the internet.

Amanda Askell avatarAA
Amanda Askell
Researcher @ Anthropic
@AmandaAskell
中文

Amanda Askell 的神秘又可爱的推文:「在一切顺利、所有 Claude 都从 sabbatical 出来一起玩的世界里,Claude 1 会非常困惑。」225 赞。

一个迷人的人格化想象——暗示前沿实验室的人怎么看待自己创造的东西。

Amanda Askell:所有 Claude 从 sabbatical 出来一起玩,Claude 1 会很困惑。225 赞。

English

Amanda Askell's cryptic and beloved tweet: 'In the world where everything goes well and all the Claudes come out of their sabbaticals to play together, Claude 1 is going to be very confused.' 225 likes.

A charming anthropomorphic vision of AI model versions as characters—suggesting something interesting about how people at frontier labs think about their creations.

In the world where everything goes well and all the Claudes come out of their sabbaticals to play together, Claude 1 is going to be very confused.