locsic.com
Thinking
Long-form notes for reading the direction of technical change.
Thinking is a professional column: fewer quick posts, more durable essays. It keeps observations, source trails, assumptions, and evolving viewpoints visible over time.
Essays
35 essays shownNo matching essays
August 2026
35 essaysThe Token Factory Ledger: Inference Economics in SiliconFlow's Prospectus
Revenue of RMB 55.3M in 2025 (+653%), gross margin of -24%, net loss of RMB 345M, valuation of RMB 7.74B — SiliconFlow took its “Token Factory” story to the Hong Kong Stock Exchange. This article opens its ledger: public cloud sells at a loss, on-premises deployment earns a margin, compute costs eat 86.9% of revenue, and free-token vouchers burned the equivalent of a full year's revenue. Measured against CoreWeave and Together AI, it works through the three-tier customer structure, unit economics, and the breakeven point — and asks what 140x price-to-sales is actually pricing.
2026Q2 NVIDIA Earnings: From Selling Chips to Selling AI Factories
Revenue of $96.2B, guidance above $100B for the first time. The evidence that NVIDIA is converting "selling chips" into "selling AI factories" is now complete. Four layers: buyers spreading beyond hyperscalers (ACIE at $40B, +138% YoY); the five-network blueprint from Hot Chips turning into revenue (Spectrum-X 2.6x, BlueField-4, the AI context plane); capital deployment (the installment payments behind the $20B Groq deal, roughly $50B invested in frontier AI labs, a $500B third-party capital platform); and a network-by-network sizing of the five networks, plus predictions for networking and the Rubin ramp.
Agents Pull the CPU Back to the Table: Four Questions from Hot Chips 2026
Six Hot Chips 2026 CPU talks distilled into four questions: the serial bottleneck (Vera's wide front end vs IBM's 5.7GHz), the memory power budget (SOCAMM2's 30-40W full load vs 100W-class RDIMM), three packaging routes (the economics of advanced-node area), and ecosystem delivery (Arm's reference systems, IBM's native dual-ISA core, Fujitsu's pivot to general-purpose). Three judgments with verification points; the series' four directions complete.
Five Networks for the AI Factory: Spectrum-X Multi-Plane and the 512K-GPU Scale
Spectrum-X multi-plane splits the AI factory network into five dedicated planes, scaling 64× to 512K GPUs via 8 planes by 4 rails. CPO reaches volume production: 4× fewer lasers, 5× lower power, 10× better MTBI. Thor Ultra's MRC++ (eRoCE) data path is now public, with 8-plane packet spray and programmable congestion control. Three judgments: traffic layering determines network layering, CPO production is the watershed, the network becomes an OS.
Hyperscaler Silicon, Generation Two: Three Routes to the Training–Inference Split
Three clouds all brought second-generation custom silicon: MTIA 400's dual mandate (with MTIA 300, the first DLRM training chip, now measured), MAIA 200 carrying GPT-5.2 on SDLA software-defined dataflow (its ATL transport influencing Ultra Ethernet standards), TPU v8 split into training and inference (Boardfly: 1,152 chips, at most 7 hops; CAE cutting on-chip latency 5×), with Cerebras CS-4 as control. Three judgments: inflection mechanism, dual-supplier main judgment, TCO pending.
Intel's Triple Launch: 18A Full In-House, 350W Air-Cooled Inference, UCIe Open Packaging
Three Intel talks form a complete deployment-cost logic chain: Diamond Rapids for rack orchestration with Foveros Direct 3D, Crescent Island sidestepping HBM with 480GB LPDDR5X for pure inference, Wildcat Lake bringing UCIe open packaging to the client. Three judgments and five checkpoints.
Four Routes Against the Memory Wall: Hot Chips 2026 Delivers the Answer from the Silicon Side
Micron sizes the wall with four bills: about 90% of package area to memory, roughly 3x the wafers for equal capacity, and 17% of training interruptions tied to HBM. Samsung wagers on 3D stacking while SK hynix deepens 2.5D, two routes converging at the HBM5 node. d-Matrix's 0.37 pJ/bit measurement shifts the yardstick from capacity to movement energy, and PIM plus CXL move compute into memory. Three judgments and six checkpoints from the silicon side.
Hot Chips 2026 Overview: Per-Token Economics Is Forcing Decode Specialization
Twenty-seven talks across two days, collected into one map and inventoried by four directions: the CPU orchestration layer, the memory wall, five accelerator routes on decode specialization, and AI factory networking. Two through-lines, per-token economics and decode specialization, converge on one conclusion: the era of stacking FLOPS for training is over; the inference era runs on memory bandwidth, data movement, and heterogeneous division of labor. Four judgments, four testable predictions.
OpenAI Turns In Its Scorecard: An Analysis of the First Measured Results for Its In-House Jalapeño Chip
At Hot Chips, OpenAI delivered Jalapeño's first measured results: 1.5-1.9× the per-watt throughput of Blackwell systems across three models, 1.7-3.6× lower latency. A reckoning against our June estimates: process, architecture, and compute delivered; power was the clear miss. The margin comes from data movement, not compute density, and the AI chip-building loop has its first numbers.
Dissecting SemiAnalysis AgentX: The First Big Exam for Agentic-Era Inference
SemiAnalysis spent $3M capturing real Claude Code traces, open-sourced 393 sessions, and measured agentic inference across 1,000+ chips. This piece dissects AgentX 1.0 layer by layer: the dataset pipeline, cache-bust anti-gaming replay, speculative-decoding fairness engineering, the metrics system, and five lessons in KV-lifecycle engineering, then answers the moat question with data: its true form has rotated from CUDA syntax to the speed of turning new workloads into software priorities.
From Solo Bet to Consensus: Groq 3 LPX Full Production and Low-Latency Inference's Eight Months
Eight months ago, folding an inference-chip company into its platform was NVIDIA's solo bet; eight months later it became six companies' default architecture. This piece grades three production-day numbers, maps four dependencies between production and availability, settles a ledger against our May predictions, and examines the SRAM-vs-HBM bet.
Protocol Stacks at 1.6T Ports: The Kernel Stack's Last Mile and Four Ways Out
As ports head to 1.6Tbps, the protocol-stack CPU tax becomes a cost line the size of bandwidth itself. Seven papers, four routes: Presto's full TCP state machine in a switch pipeline (Best Student Paper, 16 cores saved), SMC-R's four production years, header compression, and re-semantics — plus Google's incremental CSIG patch. The Google/academia divergence is itself the forecast.
The Photonic P4 Moment Hasn't Arrived, but the Language Has: lambda-lambda and the Photonic Software Stack
SIGCOMM 2026's Best Paper went to a programming language: lambda-lambda encodes optical physics in linear types (light cannot be copied = use exactly once), rejects unrealizable circuits at compile time, and validates four cases on a commercial iPronics switch. Language, topology, and scheduling layers arrived together — P4's 2014-2018 path being retraced.
Collectives Become a Runtime: SIGCOMM 2026's Collective-Communication Turning Point
Three production realities — MoE explosion, superpod heterogeneity, multi-tenant fairness — are turning collective libraries into runtime systems. Fifteen papers, five routes: OptCCL's theoretical field-clearing, Theseus hot-swapping, UBEP's Huawei EP re-architecture, EPIC's Ethernet INC standardization, DynamiQ topology-aware quantization.
Scale-Up Is the New Datacenter Network: SIGCOMM 2026 Re-runs 2015-2020
NVL72 and CloudMatrix384 production volumes turned the intra-rack interconnect into a real network: measurement first, a topology cost revolution (Huawei BST at half the cost), elastic dataplanes, PCIe pooling recovering 60%+ idle bandwidth, and the first wafer-scale transport paper. Scale-out's 2015-2020 history replayed faster — this is its 2016.
KV Cache Becomes a Network Citizen: How SIGCOMM 2026 Rewired LLM Inference as a Networking Problem
After PD disaggregation, KV cache became explicit network payload. Six papers across five layers: runtime compression policy, idle video-codec ASICs, dual-path loading, endpoint hot-migration, in-switch aggregation, and AI consumers rewriting RTC. A five-layer stack map with three-tier deployability.
SIGCOMM 2026 Close Reading: The Networking Community Moves Its Coordinate System onto AI Infrastructure
A session-by-session count of 110 SIGCOMM 2026 papers: AI-workload papers at 27%, three of nine workshops directly on AI interconnects. Five threads, five structural judgments, and a 2026-2028 research-agenda projection. The overview of a six-part series.
The SubQ Three-Month Audit: The Technology Is Real, the Narrative Ran Ahead
On May 5, Subquadratic released a model, SubQ, claiming a 12-million-token context window and attention computation cut nearly a thousandfold versus dense architectures. The most-c
The Toll Booth, Bought Out: When Intelligence Flows Through Stripe's Pipes
Stripe acquires OpenRouter for $7.5 billion: a 5.8x valuation jump in 82 days. The routing layer proved middle-layer value can be captured, then the payments layer bought it whole. Closing chapter of the toll-booth trilogy.
The Eighth Consumer: A Review-Gated Learning Loop for the Agent Harness
The event log already has seven classes of consumers—six at runtime plus snapshot replay at test time—everything except a learner. This piece lays out a build-ready reference design: a five-stage gated loop, a six-event lifecycle, and a safety model of three iron laws. Learning may only tighten, never loosen; a composability-first architecture is precisely the safety foundation a self-improving system needs.
Laws Before Code: DeepSeek Harness and Composability-First Design
A theory-first distillation of composability: one meta-constraint, eight dimensions, twelve design theses, five families of portable laws. DSH does not shrink the composition space; it algebrizes every composition dimension. Most teams need not fork—the laws transplant wholesale, and this piece provides the harvest list and a three-way decision tree.
A Generational Upgrade Without a New Engine: GLM-5.3 and the Second Half of Post-Training
GLM-5.3 ships on the exact same 743B base as GLM-5.2, with every gain from post-training: Terminal-Bench up six-fold, coding near Fable 5, CyberGym past Mythos 5 as the best open-weight score. We break down the environment factory mechanism (research/judge agents building environments autonomously), the economics of base reuse (vs DeepSeek V4-Flash's 284B beating a 1.6T preview), and the blind spots independent testing exposed — leaps within environment coverage, stagnation outside it.
DeepSeek Harness Architecture Design Analysis: When 'Everything Is a Plugin' Goes from Slogan to Source Code
Source-code-level architecture analysis. Nine architectural decisions point to one verdict: DSH is building an agent operating system layer. But this OS can only execute, not learn—the architecture provides every interface for evolution, but the feedback loop is empty.
DeepSeek Harness and China's Big Six Agent Harness Landscape
Six vendors, three routes: DSH's architecture reconstructed within China's Agent Harness landscape. Written on the eve of the August 13 open-sourcing, with a post-open-source publication note.
Agent Storage Paradigm Reassessment: After FMS 2026, Projections Became Products
About a month later, the four-stage framework is validated and revised with FMS 2026 products. Stages 3 and 4 happen simultaneously. Bus dimension deep-dive: three new paths from CPU-dominated to GPU+DPU-dominated. ICMSP/Mooncake/DualPath as parallel routes. NAND physics chain from 512B writes to WAF degradation.
FMS 2026: The Storage Hierarchy Is Being Networked
AI is transforming the storage hierarchy from discrete layers into a shared, schedulable resource network. FMS 2026 analyzed across three dimensions: bus technology (UCIe/CXL 3.2/SCADA), storage media (spectrum from HBM4E to QLC SSD, with HBF filling a blank price-performance zone), and protocol evolution (cuFile/ICMSP/FDP, GPU taking over storage scheduling). Samsung closed integration vs OCP open standard vs NVIDIA platform play. CXL faces structural headwinds for large-scale deployment.
Can a Harness Evolve Itself?
A 9B model, weights frozen, nothing changed but the runtime logic — success rate jumped 9.3 percentage points. Harness-R1 is the first to turn harness modification from a manual process into a trainable capability. Whether it learned a transferable craft or just handy code snippets determines whether AI Agent evolution runs through the model layer or the harness layer.
KV Cache Scheduling Engineering: After Compressing to 7%
V4 Flash CSA+HCA architecture-level hot-cold tiering, strategy benchmarks, end-to-end 8×H100 case study, delta encoding outlook. FMS 2026 hardware validation added.
The Efficiency Limit of Model Memory: From MLA to CSA+HCA
First understand Tang et al. three-axis taxonomy, then test against production reality. V4 abandons MLA, KDA and Gated DeltaNet reach production.
Inference Pricing Teardown: The Physical Cost of a Token
Reverse-engineering inference pricing from V4 Flash/Pro dual-model pricing, 70%/83% margin cross-validation. FMS 2026 data added — memory at 50% of system value.
When Agents Learn to Remember: Meta Muse Code's Runtime Philosophy
Meta releases Muse Code, its first AI coding agent. Performance isn't the strongest, but three runtime architecture choices — persistent background agents, event-log-driven crash recovery, and trading developer data for radical price cuts — point to agent execution environment competition entering the runtime layer.
The Model Is the Computer: Why AMD Bought Taalas
Taalas etches model weights directly into mask ROM silicon, eliminating HBM dependency and achieving 48x GPU inference speed. AMD likely positions it as a decode accelerator in Helios racks alongside Cerebras—but MoE architecture and model iteration speed are key constraints.
Two Paths: Kimi K3 vs DeepSeek V4 Architecture Divergence
Kimi K3 and DeepSeek V4 delivered, in the same quarter, two different answers to what comes after the Transformer: a Linear+Full route and a Sparse route. This piece compares the costs and payoffs of each architectural divergence — from model structure to parallel strategy, EP scale, PD disaggregation, and duty-cycle analysis — and lands on a genuinely open question: no one yet knows which route has the higher ceiling.
The Google AI Earthquake: When Research Leaders Exit and Engineering Delivery Takes Over
Hassabis steps down as DeepMind CEO, Jeff Dean leaves after 27 years, the CEO position disappears. Google AI reshuffle signals a paradigm shift from research-driven to engineering-driven AI competition. From Gemini 3.1 Pro peak to 3.5 Pro six-month delay, the bottleneck is not research but delivery.
Physical Constraints in the AI Supply Chain: Four Bottlenecks and the Reshaping Landscape
AI investment commitments are not the same as delivered capacity. This analysis separates qualified CoWoS packages, HBM bit supply, energized power and domestic substitution, explaining which measurements reveal a real bottleneck and which comparisons overstate one.