inference efficiency
2 published articles
LLMs & Models3 min read
AI Research
Penelope hides its reasoning in a single decoder layer to cut inference costs
Penelope, a latent-reasoning framework for decoder-only Transformers, localizes recurrent computation to a narrow decoder interval, cutting inference latency without a major accuracy hit. The paper describes a curriculum that shifts reasoning from visible tokens to an internal GRU loop.
2026-08-04
AI4 min read
Artificial Intelligence
The hardest lesson for AI reasoning engines: when to shut up
MIT researchers propose OS-Pruner, a plug-in that dynamically stops chain-of-thought reasoning when further computation isn't worth the token cost. Tests show 20-60% length reduction with minimal accuracy sacrifice.
2026-07-29