2026-08-26 · Reckoning update · A sequel to "Jalapeño Teardown: When an AI Company Starts Building Its Own Heart" (2026-06-27) and "Dissecting SemiAnalysis AgentX" (2026-08-25)
Prologue: Rubin Is Still on the Way, but OpenAI Turned In Its Scorecard First
Yesterday, the AgentX teardown closed on a timetable: SemiAnalysis said outright that Rubin would be running the InferenceX benchmark within August, which meant the independent check on NVIDIA's 30×/35× marketing numbers finally had a date. That piece's theme was "benchmarks are power": public benchmarks are becoming the stage where chip results get released.
The timetable hadn't delivered Rubin yet; it delivered a more surprising entrant first. On August 25, at Hot Chips, OpenAI published Jalapeño's first measured results: against NVIDIA's incumbent Blackwell systems, across three open-weight models, 1.5–1.9× the AI work per watt at peak throughput and 1.7–3.6× lower end-to-end latency. Jalapeño is rated at 700W, with measured sustained power at or below 550W; the comparison systems are rated at 1200W for GB200 and 1400W for GB300.
Two months ago, three days after Jalapeño was unveiled, we wrote a teardown that left behind an estimate table and five hard pass/fail metrics. This article does two things: read the scorecard layer by layer, then go back and settle accounts with June.
1. The Scorecard: How to Read the Four Layers
First, the test setup. The benchmark is InferenceX, SemiAnalysis's public inference benchmark, which exercises the full serving process (how its dataset is constructed and how replay is kept honest were covered in detail in the earlier AgentX teardown). OpenAI picked three open-weight models: GPT-OSS 120B, DeepSeek R1 670B (MXFP4 quantization, a block-scaled 4-bit floating-point format), and Kimi K2.5 1T (same format). The workload spec is uniform: 8k input / 1k output, STP mode (single-token prediction: ordinary token-by-token decoding, no speculative acceleration). Cross-system comparisons are normalized by each vendor's published package power rating.
The results come in four layers, each more startling than the last, and each needing more explanation than the last.
Layer one, peak throughput per watt: 1.5–1.9×. On GPT-OSS 120B (vs GB200), Jalapeño delivers 85,448 mixed tokens per second per kilowatt (a throughput metric counting input and output together); GB200 delivers 44,960. On DeepSeek R1 (vs GB300) it is 19,641 vs 11,781; on Kimi K2.5, 18,195 vs 11,862. The larger the model, the smaller the multiple (the comparison system also shifted from GB200 to GB300 at the same time, so part of the narrowing comes from the changing baseline), but every result holds above 1.5×. This layer is the most solid of the four: a clean metric, and a real gap.
Layer two, end-to-end latency: 1.7–3.6× lower. In the same battery of tests, a single full request on GPT-OSS completes in 1.03 seconds vs 1.80; on DeepSeek R1, 1.65 seconds vs 5.99. This layer is the gap an interactive product can feel directly.
Layer three, minimum time-between-tokens (TBT, the interval between two output tokens): 2.7–4.1× lower. Jalapeño pushes TBT down to 0.69–1.44 ms, corresponding to per-user output speeds of 694–1,459 tokens/second; the GB systems' best points sit at 1.87–5.90 ms. This layer is the choke point for agentic workloads: an agent must complete dozens to hundreds of steps in series, and every step's wait compounds. For "highly interactive workloads," the company separately quoted a 2.1–4.1× performance range; the appendix lists no detail behind it, so file it away as-is.
Layer four, matched-TBT throughput: 53.7×, 104.3×, 56.1×. These are the most alarming numbers in the release, and also the easiest to misread. The metric: tune each system to the best interactive quality the GB system can reach (its own previous-best TBT), and compare throughput at that point. Holding per-user speed at 169 tokens/second on DeepSeek R1, GB300 can deliver only 118 mixed tokens per second per kilowatt; at exactly the same user-experience point, Jalapeño delivers 12,258. The gap comes from general-purpose GPUs struggling to sustain throughput in the low-latency, high-concurrency region, precisely where Jalapeño's architecture can keep extracting more. These multiples measure "how many users you can still serve when interactive quality is not compromised"; any citation must carry the conditional clause, and flattening them into "100× faster" is a misreading.
One more methodological note: OpenAI normalized by rated power (700W vs 1200/1400W), while Jalapeño's measured sustained power never exceeded 550W. Computed on measured power, its per-watt numbers would look better still, unless the comparison systems also ran proportionally below their ratings, and their sustained power is undisclosed, so there is no way to run the symmetric check. The direction of the normalization is conservative for Jalapeño. Worth putting on the record.
Beyond the scorecard, the company left one line to whet the appetite: on internal frontier OpenAI models, the advantage widens further. No numbers, just a company claim, so we set it aside.
2. The Architecture: The Deciding Move Is Data Movement
The Hot Chips materials filled in the architectural detail the June piece lacked, along three main threads.
The matrix engine uses the MXFP4 numeric format with a weight-stationary systolic array, the same lineage as Google's TPU. The difference is flexibility: Jalapeño supports smaller compute dimensions, so oddly shaped matrix multiplications don't fall off the performance cliff of a large systolic array (per SemiAnalysis's analysis). This also explains how three non-OpenAI models run on it at all: the chip targets the shared structure of modern LLM inference, with GPT-specific customization as just one layer on top.
The system design is organized around the phase structure of inference. Prefill (the input-processing phase) eats compute; decode (token-by-token generation) eats memory bandwidth; communication leaves compute units idle, waiting. Jalapeño's approach: model state (including the KV cache, the intermediate data that holds already-processed context during inference) is explicitly placed and kept local, with combinations of compute, memory, and network scheduled by phase; the interconnect forms one large domain, so an entire workload stays inside a single connected system, and cross-system data movement is simply eliminated. The Register reports the system-level figures as 128 chips, 1.7 EFLOPS, and 27TB of HBM. The two numbers mesh: 1.7 EFLOPS divided by 128 is roughly 13.3 PFLOPs (EFLOPS to PFLOPs is a factor of a thousand; 1 EFLOPS = 1,000 PFLOPs), which matches SemiAnalysis's disclosure of 13.4 PFLOPs MXFP4 for the single B0 chip; 27TB divided by 128 is roughly 211GB, above the ceiling of the June estimate (the company has not disclosed per-chip memory separately; this is a back-calculated value).
The chip-level comparison hides this round's most important technical signal. According to SemiAnalysis: the B0 tape-out (the optimized stepping after A0) has a single compute die close to the reticle limit (the ceiling set by lithographic mask area), built on TSMC N3P, at 13.4 PFLOPs MXFP4; Rubin's compute die, same process and similar size, delivers 17.5 PFLOPs NVFP4 (NVIDIA's 4-bit floating-point format). Run the numbers, and Jalapeño's paper peak density is roughly a quarter lower (about 24%), yet its measured throughput per watt comes out ahead. The difference traces to data movement and phase-aware scheduling: it computes somewhat slower, but it moves data and idles far less. That is a correction to the June piece: back then we chalked the efficiency edge up mainly to process dividends and low-power design; what actually delivered was the architecture's fit to the shape of the workload.
A side note on where the field divides. The Register's coverage named Taalas: the route of betting performance on one specific model while sacrificing programmability has been falsified. With results from three external models, Jalapeño has drawn itself onto the other side of that line.
3. Settling Accounts with June: The Estimate Table and the Five Metrics
The June estimate table, item by item against August measurements:
Power is the only outright miss, and the cause is worth recording: the June estimate assumed an air-cooling-friendly 300–500W envelope, and OpenAI pushed the power wall to 700W, so the premise didn't hold. The direction is actually favorable: a 700W rating is still only half of GB300's, and 550W measured is lower still, and the compute density packable per rack went up instead. June also projected ~29 TFLOPS/W of per-card paper efficiency, "the highest ever"; this round's measured metric is effective throughput per watt. The two cannot be settled against each other directly; directionally, they are consistent.
The five hard pass/fail metrics, two months on:
- Deployment timing: in progress. The official line remains that deployment begins at the end of 2026, with Richard Ho adding that volumes will be "very small"; SemiAnalysis's shipping model has production ramping quarter by quarter through 2027, with Q4'27 as the main output.
- "50% lower cost": on firmer footing. In June this was still Hock Tan's TCO promise; now it is backed by public benchmark numbers at 1.5–1.9× per watt. Marketing language has become an auditable engineering figure, though throughput per watt is not TCO, and the real verdict still waits on 2027 financial-report metrics.
- Software ecosystem: partial evidence. Three open-source models that were never in the plan were brought to high performance within two months, which says something about architectural flexibility; day-to-day PyTorch/Triton pipeline support remains unverified.
- HBM and capacity: B0 being in the fab is a positive signal; 2027–28 supply remains to be proven.
- Generational cadence: Gen 2 is deep in development and Gen 3 is taking shape. B0's roughly 25% per-watt improvement over A0 (per SemiAnalysis's data) is the first physical evidence for the "one generation per year" promise.
The June piece asked, in its software-adaptation section: "The hardware taped out on schedule — can the software be polished before large-scale deployment at the end of 2026?" The most unexpected thing in the August scorecard is precisely a clue to that answer: the software was written by AI. Next section.
4. Independent Verification: SemiAnalysis's Cross-Analysis
All results so far are OpenAI's self-testing and self-reporting. InferenceX is a public benchmark, but the party that ran this round of tests was still OpenAI itself (The Register's wording: presumably unofficial). Independent verification comes from the analysis letter SemiAnalysis published the same day; two conclusions are worth excerpting.
First, Jalapeño's STP output throughput per MW exceeds the MTP figures Vera Rubin published this July (MTP: multi-token prediction, the speculative-decoding technique that generates several tokens at once). In plain terms: Jalapeño, without speculative decoding, beats Rubin's public numbers with speculative decoding; the margin over GB200's 2025 MTP results is larger still. SemiAnalysis's method compares immature system against immature system, to avoid dressing up a maturity gap as a hardware gap: Rubin's July 2026 numbers against GB200's 2025 numbers, so as to align the early bring-up stage (the phase when a new system is first made to run) and level out software maturity. Jalapeño taped out after Rubin; neither side is mature, and both sets of numbers will keep rising.
Second, the timeline. All public results were taken from A0 silicon; SemiAnalysis's exact words are "just 9 months into the program," and B0 is already in the fab. Measured against Rubin's timeline, Jalapeño reached today's public results in far less time; SemiAnalysis's word for it is "shockingly quick."
These two conclusions move this round's credibility up a notch, but the self-testing discount still applies: model selection, system configuration, and test execution are all in OpenAI's hands. When Rubin officially joins InferenceX, same-benchmark, same-stage numbers from both vendors will be the final verdict.
5. AI Builds Chips: The Loop Gets Its First Numbers
The June piece judged that "the coupling between models and silicon will keep deepening." The August scorecard turns that judgment into an evidence chain with numbers attached.
On the design side, OpenAI's models accelerated design exploration, the verification loop, and arithmetic-circuit optimization, compressing design-to-tape-out to 9 months. On the programming side, using Codex together with GPT-Astra, three unplanned open-source models were brought to high performance within two months. The weightiest number: for selected attention and MoE modules in GPT-OSS, the AI-generated implementations run 1.5–1.8× faster than the versions written by human experts. The company attached its own qualifier: selected modules, not the full model.
There is also one easily skipped sentence at the architecture level: Jalapeño was designed to be a legible programming target for both humans and AI: local tensors, explicit communication, predictable synchronization. Parallel programming, the classic hard problem, is here decomposed into mapping, placement, and scheduling problems that AI can solve. Chips designed for AI; AI writing programs for the chips; and the efficiency that comes out feeding back into the design of the next generation of chips. The speed at which this loop spins was something June's "hidden bet" section did not anticipate.
6. Four Soft Spots and Three Judgments
As usual, first spell out where this article could be proven wrong:
- Self-tested, self-reported. No third-party execution record this round; the choice of models and configurations rests with OpenAI.
- Generational mismatch in the comparison. The control group is Blackwell. Jalapeño's scaled deployment comes in 2027, by which time Rubin and AMD's MI455X are both expected to be ramping production in early 2027. Ho himself conceded at the media briefing: by the time Jalapeño reaches full deployment, the competition may have advanced significantly.
- Between small volumes and scale lies 2027. Very small volumes at the end of 2026; Q4'27 is the main wave. The pitfalls of yield, supply chain, and large-scale operations are all still ahead.
- Training still runs on NVIDIA. The company explicitly commits to continuing broad deployment of NVIDIA and other partners' accelerators. What this chip rewrites is the cost structure of inference; the other half of the compute map is still staked on the NVIDIA camp.
Three judgments:
OpenAI's economics has its first measured evidence. Useful work divided by electricity is the hardest metric of inference cost. 1.5–1.9× per watt means the same power budget serves 50–90% more volume; the 700W-versus-1400W difference is a second ledger: more compute fits per rack, and the cooling and power-distribution burden is nearly halved. The company's product-side translation is blunter: inference efficiency in ultra-fast mode now reaches the level that used to belong to fast mode alone. The operating-leverage narrative (revenue growth outpacing cost growth) has benchmark numbers underneath it for the first time.
The clock on NVIDIA's window now has numbers on it. The training-side cushion is intact, but the erosion of inference share now has a date: from Q4 2027, the biggest buyer on the customer list starts answering the cost question with its own silicon. For NVIDIA's valuation narrative, this cuts more directly than any cloud vendor's ASIC.
"Benchmarks are power" got a validation within 24 hours. OpenAI ran its first measured results to InferenceX's public specification and published them, appendix and line-item detail included, on its own blog, and SemiAnalysis ran an independent cross-analysis the same day. When a vendor's debut results choose a third-party benchmark as their release venue, the benchmark's neutrality becomes infrastructure. The thesis from yesterday's piece landed less than a day ago and already has its first citation case.
The June piece read Jalapeño as the latest coordinate point of the "models + first-party silicon" full-stack paradigm, and deliberately declined to over-read it. The August scorecard gives that coordinate its first non-marketing evidence: the chip really exists, the performance lead is real, the roadmap is really advancing. And it is only the first. B0 is still in the fab, Rubin's same-stage numbers are still owed, and the answer on scale arrives in 2027. OpenAI has turned in its scorecard; the grading has just begun.