And in yet another instance of the AI comedy, weight decay was invented to control overfitting, but what it really does is improve training loss.
MACHINE LEARNING
-
Looped Transformers: Frozen Checkpoint Inference Optimization
By
–
Another cool research on Looped Transformers They ask the question: "Can we loop a frozen, off-the-shelf checkpoint directly at inference time without any modifications?" So naive repetition pushes hidden states outside the distribution later layers expect, so performance
-
Language Models Sleep: Context Replay for Deep Reasoning
By
–
"Language Models Need Sleep" Instead of thinking longer at answer time, this paper makes LLMs sleep before forgetting. They replay old context, write it into fast weights, clear the KV cache, and answer later at normal speed. More sleep improves deep reasoning over long and
-
Using External AI Agents with MagicPath, Cursor, and Claude Code
By
–
Here are all the instructions on how you can use external agents in MagicPath with Cursor, Codex, Claude Code, etc.
-
New Method Significantly Reduces Compute Demands for One-Shot LLM Training
By
–
One-shot LLM training demands reliable scaling laws to predict model behavior, but current scaling techniques are compute-intensive. New research introduces a method that reduces training demands significantly, lowering the time and cost of scaling:
-
Is AI Making Our Brains Weaker?
By
–
Is #AI Making Our Brains Weaker?
by @markhamh @time Learn more: https://
bit.ly/4dGNUqu #ArtificialIntelligence #MachineLearning #ML -
Frontier AI Models Struggle with Long-Horizon Legal Tasks, Study Finds
By
–
Initial LAB results from Harvey put a number on something we see across specialized AI work: under rigorous all-pass standards, frontier models complete fewer than 10% of long-horizon legal tasks, and no single model leads across practice areas. General capability isn't
-
Agent Capabilities and Permissions Should Evolve with Sandboxing
By
–
New on the Engineering Blog: The access and permissions we grant agents should evolve with their capabilities. In our own products, we set these parameters through sandboxing, which limits the scope of any potentially destructive actions. Read more:
-
ReLU Activation Function Gradient Flow Design
By
–
It is easy enough to make your own, but I think standard relu should have been defined as passing the value at zero, so gradients flow backward through it, allowing some things to be zero weight initialized when symmetry breaking isn’t an issue.
-
Constrained Generation and Memory Patterns in AI
By
–
That progression is a clean way to frame it. Constrained generation, constrained tool calling, constrained memory. Same pattern applied one layer deeper each time. The temporal piece is underrated. Most teams get to typed entities and stop, but fact invalidation is what keeps