Think the big Q here is, if all this stuff stays on device while also personalizing, where it's actually trained/updated before loading into memory. That has much grander implications beyond just inferencing it.
@mattlynley
-
On-Device AI Model Loading and Memory Management
By
–
Also if this is all happening on-device there's a model loaded in memory here.
-
Apple’s Transformer Model for Autocorrect on Neural Engine
By
–
Autocorrect/Dictation now leverages a transformer-based model using Apple Silicon/Neural Engine… with no info on how it's inferenced. Is this an Apple-based LLM from scratch or something fine-tuned? Or something diff altogether? (Knowing the company it's probably the former.)
-
Consumer Updates Announced at Developer Conference 2026
By
–
Man there a lot of consumer updates for a developer conference here
-
LLaMA Inference on Mac Studio with 192GB Memory
By
–
You can literally inference LLaMA. And that 192GB in the new Mac Studio is way more than enough to go beyond the 13B param model.
-
Apple CPU gains 20% boost for machine learning applications
By
–
So I know everyone loves to stare down the GPUs here but like a 20% CPU bump on the new Apple CPU is pretty impressive given how much you can do on a CPU these days for ML
-
Meta LLaMA regulation policy and foundation model access trends
By
–
Whether that's good or not depends on your perspective, but the numbers are constantly going up. We'll see if that pushes Meta to do something more permissive with LLaMA, or how OpenAI and other foundation model developers asking for increased regulation react.
-
Model Performance Comparison: Falcon Leaderboard Ranking Analysis
By
–
That's probably not the case! Different models perform well at different things, and the harness is an *average* for a reason. But it's notable just how much effort went into highlighting that Falcon topped said leaderboard.
-
Leaderboard obsession: Why 0.1 point differences mislead AI evaluation
By
–
Leaderboards are *a thing* in *every industry.* I can tell you as a journalist we all have an obsession with leaderboards. But we'll probably hear a lot more going forward about how Model A beats Model B by 0.1 points in this eval model, so A is obviously obsolete.
-
Falcon’s Cultural Impact on Open Source Model Competition
By
–
But Falcon might also have a separate cultural impact crater. It aggressively highlighted that it topped the Hugging Face leaderboard in terms of average performance on the Eleuther AI model eval harness. And we are probably going to see a lot more of that in OSS going forward.