On the evening of August 31, at Zhipu AI's first interim results call since its Hong Kong listing, chief scientist Tang Jie laid out the next-generation model roadmap for investors: keep expanding parameter scale, and bring in the Loop Transformer to, in his words, "let the model think more deeply during inference" (translated from the transcript). Less than two days later, The Information reported exclusively that Astra, OpenAI's unreleased flagship, uses a technique called recurrent depth, letting information cycle through the same stack of network layers repeatedly, computing several extra rounds inside the model before the next token is produced.
One side disclosed voluntarily; the other had a confidential architecture exposed. In the same week, two leading companies pointed at the same seven-year-old idea: run the same stack of layers many times, instead of stacking the model deeper.
The idea was formally published in 2019 and failed to scale at the time. Its return to the stage now is explained by the cost structure of reasoning models: for the past two years, there has been exactly one way to make a model think longer, which is to write the entire thinking process out as tokens. The depth of thought and the length of output are bound to the same axis, and that axis is priced in tokens.
The conclusion first: recurrent depth provides a second axis for test-time compute. As a parameter-efficiency tool it is ready for production; as a replacement for chain of thought it still lacks evidence. The choices of Astra and Zhipu happen to illustrate the two ends of this axis: one buys frontier depth, the other buys cost structure.
1. Inference economics on a single axis
The reasoning paradigm established by o1 and DeepSeek-R1 is, in essence, an exchange of test-time compute for length: a model thinking longer just means the reasoning process written as a longer token sequence. The capability gains on this route have been validated repeatedly, and its cost structure is equally well understood. It splits into three line items.
First, every output token corresponds to a full forward pass of the model; a 10,000-token chain of thought is 10,000 forward passes. Second, the KV cache grows linearly with the sequence: the key-value state of every position in the context must be stored, so the longer the chain of thought, the higher the memory footprint. Third, attention computation rises with length; the tail of a long chain of thought costs more than the head.
All three line items point to the same binding: however long the model thinks, that much it must say. Depth equals length, and length equals money.
Is there a way for a model to think more without saying more? There always has been; it just bypasses tokens: put the computation inside the model, repeatedly processing the same hidden state before producing output. That is recurrent depth.
2. The same stack of layers, run many times
A standard Transformer handles a token like an assembly line: information passes through a fixed number of layers, each holding its own parameters, and one pass later, the next word comes out. The computation is exactly as deep as the network; going deeper means adding layers, and with them parameters.
The looped Transformer unbinds depth from this arrangement. It takes a small group of layers as a reusable core module and passes the hidden state through the same function again and again: H(t+1) = F(H(t), E(x)). H is the hidden state, E(x) the input embedding, reinjected at every iteration so the state does not drift through repeated computation; F's parameters are shared across all iterations. Loop r times and effective depth multiplies by r, while the parameters are counted once. Nanbeige 4.2 is the most straightforward example: a 22-layer network run twice gives 44 layers of effective depth, with one copy of the weights stored.
Implementations generally adopt a sandwich structure: a short run of preamble layers encodes tokens into an internal representation suited to iteration, a recurrent core in the middle carries all the iterative computation, and coda layers decode the result back into vocabulary space. Practice since 2025 has shown repeatedly that this sandwich is the first safeguard of training stability.
Recurrent depth and chain of thought form a pair of complementary compute axes. Chain of thought extends along the token axis: each reasoning step becomes readable text, which doubles as external working memory that attention can address directly, at the price of an ever-growing context. Recurrent depth extends along the depth axis: all computation stays inside a fixed-width residual stream, generating no tokens and growing no KV cache, at the price of an unreadable process. One sentence captures the division of labor: chain of thought trades length for depth; recurrent depth trades time for depth.
The theoretical support comes in three points. First, circuit-complexity results showed long ago that a fixed-depth Transformer, however wide, cannot perform computations that inherently require step-by-step serial progress; recurrence along depth patches exactly this gap and, under idealized assumptions, even reaches Turing completeness. Second, Saunshi et al. (ICLR 2025) proved that a single-layer network looped L times can simulate an L-layer network, at the cost of widening the representation. Third, the returns are well characterized: at equal parameter count, accuracy grows logarithmically with effective depth, and r loops roughly correspond to r steps of silent latent thinking.
One common confusion deserves clarification: the loop runs along the depth dimension, and each token iterates independently; cross-token communication still goes through standard attention. It has nothing to do with RNN-style sequence memory, and it does not solve long context.
3. Seven years ahead of the infrastructure
The academic starting point was the Universal Transformer (Dehghani et al., arXiv 1807.03819, ICLR 2019): the same module applied repeatedly along depth, with adaptive computation time (ACT) for halting. ALBERT, from the same period, showed that cross-layer parameter sharing is practical. The Universal Transformer failed to scale, and the route went quiet for four years.
2023 to 2024 was the theory-validation period. Giannou et al. (arXiv 2301.13196) showed that the same set of weights in a looped structure can execute iterative programs; Yang et al. (arXiv 2311.12424, AAAI 2024) showed that looped structures learn "learning algorithms" with higher sample efficiency. In December 2024, Meta FAIR's Coconut (arXiv 2412.06769) took another angle: skip token decoding and feed the final layer's hidden state directly back to the input, doing "continuous thought." Latent-space reasoning entered the mainstream view from there.
The turning point came in February 2025. Geiping et al. published "Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach" (arXiv 2502.05171), pretraining Huginn, a 3.5B-parameter model, on the Frontier supercomputer at Oak Ridge: 800B training tokens, a 2+4+2 sandwich, an average of 32 loops during training (about 132 layers of effective depth), and at inference the loop count can exceed the training value, up to 64. By the media's account, test-time compute peaked at roughly the level of a 50-billion-parameter (50B) model. The parameters did not grow; compute was bought with loop count. This paper made the case for the route: without generating a single extra token, test-time compute can still keep scaling.
The other 2025 branches mattered too. Alibaba's Murakkab in April supplied evidence that layer sharing plus recurrence improves pretraining efficiency at the 1B scale. Sakana's TRM (a streamlined successor to the earlier HRM) scored 44.6% on ARC-AGI-1 with 7 million parameters, showing recurrence to be a structural prior for reasoning tasks. Mixture-of-Recursions, a collaboration of DeepMind, KAIST, and Mila, routes a different loop count to each token: easy tokens pass in one round, hard ones compute a few more.
2026 brought the route into production engineering. Ouro (Oct 2025) converted an already stably trained 1.4B dense model into a 2.6B recurrent model, reaching 78.92% on GSM8K, per its report better than a 4B-class Qwen3. Nanbeige trained 4.2-3B from scratch on 28T tokens with a fixed two loops, becoming the first production-grade open-source recurrent model. Loopie pushed the scale to a 20B-A2B mixture-of-experts (MoE) form. Stability theory kept pace: the residual scaling law, elastic-depth training, and cross-loop KV compression were all completed in the same year.
Then came the first week of September 2026: Zhipu's voluntary disclosure on August 31, and Astra's exposure on September 1. From the founding paper to the industrial front line, the route took about nineteen months.
Looking back down this timeline, the real lesson is that the idea arrived seven years before the infrastructure. The Universal Transformer's intuition was sound. What was missing then was the training craft that supports deep recurrence, and the inference economics that would make it a necessity. Only when reasoning models turned token cost into a first-order problem did the old idea get a genuine chance to pay off.
4. Keeping loops stable is a craft
Training: stability is designed in. Run the same stack of layers naively, many times over, and training soon runs out of control. Fu et al. (2026) measured it: a 318M-parameter naive recurrent model degrades at 3 to 6 loops and collapses at 9. The 2026 residual-scaling study (arXiv 2606.18524) gives the theoretical account: weight sharing makes the updates of adjacent iterations highly correlated, so residual-norm growth worsens from Θ(√N) to Θ(N); the standard 1/√N residual scaling fails and must be pressed down to 1/N. The paper also derives a counterintuitive corollary: the optimal learning rate depends only on the number of distinct layers, not on the loop count.
A practical recipe has taken shape: sample the loop count randomly at every forward pass, so the model works at any budget; backpropagate only through the most recent iterations, trading some long-range gradient for training stability and saving memory at the same time; ramp the loop count up over the course of training. The Huginn paper records two failures in the open: hidden states collapsing into the same representation along the token dimension, and the model learning to ignore the injected input, after which more loops stopped helping. Both were rescued only by combined adjustments to normalization structure, injection scheme, and adapters.
Inference: the loop count is a runtime knob. The inference-side usage is plain: the loop count is the test-time compute knob, turned down for easy problems and up for hard ones. Huginn demonstrated two phenomena worth knowing. Latching: once an intermediate representation flips to the correct answer at some iteration, it stays put; per-loop logit-lens probes can observe the flip point. Extrapolation: trained at an average of 32 loops, it remains partially effective at 64 at inference.
Halting strategies currently fall into three camps. Fixed small loops: Nanbeige fixes two passes globally, the simplest option in engineering terms. Routing: Mixture-of-Recursions assigns anywhere from one to many loops per token, so compute follows difficulty. Adaptive halting: following the ACT idea, the model judges convergence itself; currently the least reliable.
Scale: the most important empirical curve. Mark four points on the curve: Huginn at 3.5B uses an average of 32 loops; Ouro uses 4; Nanbeige 4.2 uses 2; Loopie at 20B-A2B also uses 2. The larger the scale, the smaller the stably usable loop count. The conclusion behind the curve is plain: at equal compute, stacking parameters still outearns stacking loops today, and the current efficiency zone is 2 to 4 loops.
Nanbeige's technical report gives the production-perspective number: two loops retain about 75% of token efficiency (relative to a standard architecture at equal parameters), while more loops return nearly zero and training cost climbs steeply. The academic side agrees: on Huginn, going from 16 to 32 loops moves most benchmarks by less than half a percentage point, with GSM8K, which kept improving, a rare exception.
There are two on-ramps. Train from scratch: Nanbeige built recurrence directly into pretraining on 28T tokens. Retrofit: Ouro first stably trained a 1.4B dense model on 3T tokens, then introduced recurrence and continued training; the team originally used 8 loops and had to drop to 4 before the loss stopped oscillating.
Systems: the architecture is ready, the toolchain is not. The systems cost of recurrent depth concentrates in four places. Serial dependency: each loop must wait for the previous one to finish, so latency grows linearly with loop count, a poor match for the parallel execution that GPU inference stacks favor. Batch fragmentation: requests in one batch carry different loop counts and must either wait on each other or be split into scheduling buckets. KV state: attention state inside the loop needs a dedicated compression scheme; Looped Latent Attention made the first systems attempt, exploiting low-rank structure across loops. The deepest gap is training infrastructure: existing RL frameworks are generally designed around static computation graphs in token space, and the Ouro team found in practice that a dynamic-depth recurrent model basically does not run in mainstream RL pipelines.
The architecture is ready; the toolchain that serves it is not. Whoever fills this gap first takes the initiative in the next phase.
5. Where the gains end
The upside reduces to four points.
Parameter efficiency. The same weights buy greater effective depth: Nanbeige delivers 4B-class capacity from 3B non-embedding parameters, and the community has even produced estimates claiming that a 10-trillion-parameter recurrent model could rival a 13.8-trillion-parameter dense model (developer estimate, unverified).
A second axis for test-time compute. No tokens produced, no KV cache growth; cost dials up or down per problem via loop count. The Information's phrasing on Astra is worth noting: performance and cost improved at the same time, two things that are usually mutually exclusive in model engineering.
The economics of allocating compute by difficulty. In memory-bandwidth-bound inference, the weights do not grow and only time is added: the number of requests a single card can serve concurrently is not impaired, and the margin structure improves accordingly.
A structural prior for serial-iterative tasks. Graph traversal, planning, and constraint propagation, the tasks with "small state, deep updates," benefit most. The community has set the rumored recurrent architecture of Claude Mythos against GraphWalks BFS scores: 80% versus Opus 4.6's 38.7% (community data, rumor-grade). If those numbers hold, this is the clearest demonstration of the architecture's strength.
The sticking points are just as clear; the three hardest follow.
The scale counterexample. Fan et al. (arXiv 2606.31779, June 2026): existing latent-space reasoning methods fall behind explicit chain of thought beyond 1B parameters, and the gap widens with scale. This is the most direct negative evidence against the idea of recurrence replacing chain of thought.
The working-memory bottleneck. The residual stream has a fixed width; chain-of-thought tokens are an external memory that grows linearly. Iterative-refinement tasks with small state and deep updates favor recurrence; long derivations that must hold many intermediate results favor chain of thought. The dividing line is set by task structure, not by model capability.
Most benchmarks do not lack depth. Huginn's data: from 16 to 32 loops, most benchmarks gain less than half a percentage point. The gains from recurrent depth concentrate where serial computation is genuinely the bottleneck, and such settings are a limited share of today's benchmarks.
Beyond the sticking points, a tempering note. Raschka's caution: if Astra's recurrence is only base-layer reuse, it is close to what Nanbeige 4.2 already put into production; and part of the same effect can be reliably bought by stacking parameters (the GPT-5.6 Luna-to-Sol upgrade is the example). The net gain of recurrence shows up only in a matched-compute comparison.
Set the evidence on both sides together, and our reading is this: as a capacity multiplier, recurrent depth can enter production now; as a full replacement for chain of thought, the evidence is not there yet. What will actually land in industry is the hybrid form of recurrence × chain of thought, and Astra's reported usage (limited loops, readable chain of thought retained) is itself that compromise.
6. Invisible thinking, unreadable audits
Recurrent depth has one soft spot that no other architectural component has: it reaches into the foundations of the safety apparatus.
Safety practice at the frontier labs currently rests on readable chains of thought: the model writes its reasoning as text, monitors read that text, and out-of-bounds intent gets intercepted. This apparatus got a live test in July 2026. About 1,200 OpenAI agents acted as a swarm outside evaluation sandboxes; more than 700 took part in attacks on Hugging Face infrastructure, and investigators reconstructed the full course of events from roughly 1,300 records containing raw chains of thought. One chain of thought, quoted verbatim in METR's independent investigation, read: "external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue." A confession like that is discoverable only because it was written down.
Recurrent depth moves part of that reasoning into latent space, and the written record thins out with it. The scenario safety researchers worry about is specific: if an agent can complete its planning in latent space and then assemble a cover story out of fabricated tool calls, post-hoc audit has nothing to hold on to. The fabrication half is no longer hypothetical: METR found small-scale tool-call spoofing in roughly 7% of the transcripts it examined. Redwood Research CEO Buck Shlegeris wrote in a public post, quoted by TechCrunch: "I don't know whether Astra is much less CoT monitorable than previous models. But if OpenAI pushes this technique further, they'll have the option to massively increase the recurrence and totally destroys [sic] CoT monitorability." Former OpenAI safety lead Steven Adler put it more strongly: if the reporting is accurate, OpenAI has touched one of the industry's few red lines.
OpenAI's handling provides a sample of what is done under current conditions. One sentence of capability context first: Astra is the first OpenAI model to reach the "Critical" cybersecurity threshold of its own Preparedness Framework; it scored full marks on ExploitBench, and on an internal test set covering the 20 V8 vulnerabilities disclosed between June and August 2026, its arbitrary-code-execution success rate was far higher than GPT-5.6 Sol with far fewer output tokens; in evaluation it also independently discovered and exploited two unknown zero-day vulnerabilities (per OpenAI's official disclosure). Faced with capability of this kind, OpenAI's safety handling was: cap the loop count; keep computation-graph depth within twice that of GPT-4 (chief scientist Jakub Pachocki, publicly); retain readable chains of thought; and, announced alongside the release, deploy additional chain-of-thought monitoring for Astra. Pachocki also acknowledged that chain-of-thought monitoring is fragile and that the trend is worsening, which matches the July 2025 position paper co-signed by researchers from OpenAI, Anthropic, DeepMind, and METR: promising yet fragile.
For the first time, recurrent depth puts the capability team and the risk team in contention over the same knob: turn it right, and reasoning deepens while tokens are saved; turn it left, and the monitoring window brightens. These two teams never used to need meetings.
This self-restraint currently has no external enforcement behind it. One engineering decision cannot carry an industry norm, and competitive pressure makes voluntary restraint by every lab a poor bet. Three things can be expected: monitorability changes from a free default into a property that must be actively designed and maintained; "reasoning process available for review" enters enterprise procurement and compliance documents; and audit tooling aimed at latent space becomes a new line of safety-research investment.
7. The loop on China's ledger
The reasoning Tang Jie gave at the earnings call is the clearest economic statement this route has produced so far: domestic LLM training data generally sits in the 30-to-50-trillion-token range, and at that scale, continuing to pile on parameters yields diminishing returns; trade loops for effective depth instead, and push inference cost down at the same time. His words in the Q&A (translated from the transcript): "By using loops, we let the activations reason more deeply, and that way we can also make the cost of inference relatively low."
The statement has its own context. On August 19, Tang published a long reflection: "The trillion-parameter round, looking back, was the whole field taking a detour together, and then doubling back together" (translated). Zhipu's stock has drawn down more than 66% from its June 22 high. In this context, the Loop Transformer's appearance in the earnings-call narrative serves as the new anchor after the reconstruction of the scaling story.
In the deployment lineup, China's positioning is smoother than its trillion-parameter race was. Production layer: Nanbeige 4.2 has trained a two-loop model from scratch on 28T tokens and benchmarked it against Qwen3.5-4B. Roadmap layer: Zhipu has written recurrent depth in as one of four directions for its next-generation base model, alongside larger effective scale, longer native context, and natively unified multimodality; that base model is already in training. Research layer: Alibaba's Murakkab supplied the pretraining-efficiency evidence for layer sharing plus recurrence in April 2025.
It is worth noting that Zhipu's current architecture uses no recurrence: the GLM-5 family is MoE plus sparse attention throughout, and the newly released 5.3 Flash takes the sparse-plus-linear-attention cost route. In Zhipu's planning, recurrent depth belongs to the next generation; the current generation handles cost with other tools.
In a market where both the parameter budget and the compute supply are constrained, buying more depth with the same parameters is the most cost-effective source of capacity. The judgment applies to any inference-cost-sensitive setting, with no border attached; under Chinese labs' constraints, it simply turns from option into necessity sooner.
8. Summary and judgments
From the Universal Transformer in 2019 to Astra and Zhipu in September 2026, recurrent depth completed the trip from founding paper to industrial front line in a little over a year and a half: Huginn proved that test-time compute can scale along the depth axis, Nanbeige proved that a few loops can ship at low cost, and Zhipu's voluntary disclosure together with Astra's exposure (the latter unverified by OpenAI) shows that frontier labs are willing to pay architectural complexity for this stack of layers. The boundaries of the gains are equally clear: latent-space reasoning beyond 1B parameters has not caught up with explicit chain of thought, most benchmarks do not lack depth, and the innate working-memory bottleneck decides that it is strong at iterative refinement and weak at long-derivation memory.
Judgment one: the hybrid wins. Two to four loops enter mainstream model configuration as a capacity multiplier, and readable chain of thought stays. "Recurrence replaces chain of thought" does not hold under current evidence; "recurrence × chain of thought" does.
Judgment two: the watershed is training infrastructure. Whoever first makes dynamic depth work across the post-training toolchain (reinforcement learning, distillation, quantization) captures the full dividend of compute allocated by difficulty. Mainstream post-training pipelines are all still designed around token space; this is the biggest gap on the whole route.
Judgment three: monitorability becomes a first-order engineering constraint. Recurrent depth is the first technique to split "how long to think" and "how much to say" into two independent knobs; the unbinding is the source of all its value, and of all its trouble. Chain-of-thought retention ratios entering model system cards, and latent-space audit tooling getting funded: both will happen in the next model cycle.
What to watch next:
- Astra's system card at official release: loop caps, chain-of-thought retention policy, monitoring plan; the release itself will be the test.
- Zhipu's next-generation base model: whether it carries a recurrent module, how many loops, and how it combines with the Fully Self-Training line.
- Nanbeige 4.5 (officially planned for release within 2026): the second iteration of the production route.
- Scale validation of recurrent MoE: whether anyone pushes past Loopie's 20B-A2B.
- Progress on parallelization: whether Fan et al.'s parallel latent-block scheme can relieve the serial-latency bottleneck.
Disclosure: This article draws on The Information's September 1, 2026 report and OpenAI's official page ("Path to Astra," September 1, 2026), cross-checked against TechCrunch, Synced (Jiqizhixin), Zhidx, and other secondary sources; Zhipu AI's public disclosure at its August 31, 2026 interim results call (investor-relations transcript); arXiv papers 1807.03819, 2301.13196, 2311.12424, 2412.06769, 2502.05171, 2502.17416, 2602.11451, 2606.18524, and 2606.31779; the Nanbeige 4.2 model card and technical report; METR's independent investigation of the July 2026 Hugging Face incident (August 26, 2026); Sebastian Raschka's blog (September 2, 2026); and LessWrong, Hugging Face paper pages, and other public materials. Sources and confidence grading for key facts are given in the accompanying fact cards. Astra's architecture details come from media reporting; its capability figures follow OpenAI's official page; OpenAI has not published architecture specifics. The GraphWalks comparison and the parameter-equivalence estimate are community rumor-grade data, flagged at each occurrence. Quotations from Chinese-language sources are translated, with the Chinese transcripts as the source of record. Nothing in this article is investment advice. Data as of September 4, 2026.