Recursive Language Models

Recursive Language Models (RLMs) are an emerging architecture that extends large language models with the ability to reason recursively—breaking complex problems into subproblems, solving each independently, and composing the results. Where standard autoregressive models generate tokens left-to-right in a flat sequence, RLMs can decompose and recompose at multiple levels of abstraction.

The core insight is that many reasoning tasks are inherently hierarchical. Writing a complex document involves structuring sections, then paragraphs, then sentences. Solving a math problem requires decomposing it into sub-problems. Planning a multi-step task means nesting goals within goals. Standard transformers handle this implicitly through attention patterns, but RLMs make the recursion explicit—allowing the model to "call itself" on sub-tasks and integrate the results.

This approach addresses a fundamental limitation of current LLMs: the difficulty of consistent, deep compositional reasoning. A model generating a 10,000-word document in a single forward pass must hold the entire structure in its context window simultaneously. An RLM can instead plan the high-level structure, then recursively expand each section, maintaining coherence through hierarchical composition rather than brute-force attention.

RLMs connect to broader themes in AI architecture. They echo the structure of recursive programs in computer science, the hierarchical planning used in AI agent frameworks, and the compositional nature of human cognition. As reasoning models push toward more complex autonomous tasks, recursive architectures may prove essential for scaling beyond the current limits of flat autoregressive generation.