And so it begins. Looks like / I hope there will be some fresh open-weight models to analyze and write about soon… It's been a while!
@rasbt
-
AGI Reached When Experts Trust Models With Email Access
By
–
In fact, my measure of when AGI is reached is when most experts trust a model enough to give it access to their email.
-
Rebranding Apple Foundation Model naming concerns and alternatives
By
–
Yeah, agreed. I think it's common (like Composer). I was more thinking of the term "Foundation" in "Apple Foundation Model" when the foundation really is Gemini.
I would probably call it sth like Apple Atlas or so (but I am not a marketing person :D) -
Apple Foundation Models Based on Google Gemini: Naming Questions
By
–
> Apple Foundation Models will be based on Google's Gemini models and cloud technology. This is the right move! But will they still be called Apple Foundation Models if they are Gemini fine-tunes?
-
Llama 4 Architecture and NoPE Ablation Studies Analysis
By
–
No worries. And in this case, you are right but that's a smaller architecture, not a Llama 4 sized one trained from scratch. Otherwise, the original NoPE also had ablation studies
-
Smaller Architecture vs Full-Scale Llama 4 Training from Scratch
By
–
Ok, but that's a smaller architecture, not a Llama 4 sized one trained from scratch. Otherwise, the original NoPE also had ablation studies
-
Llama 4 Memory Loss After Extended Time
By
–
tbh it’s been a while since Llama 4, I might have forgotten
-
Skepticism on MIT Paper Production Claims and Memory Limitations
By
–
1) The paper was just published 2 weeks by MIT researchers, did DeepMind really already put it into production? I can't believe that's true. 2) I think it's a great paper and promising method, but perfect memory is a bit far fetched. It's essentially just chunking up the
-
RoPE vs NoPE: Ablation Studies in LLM Architecture Design
By
–
I think the problem is that we don't have ablation studies: how would the same architecture and training run do with RoPE vs NoPE?
That being said, I'd say Kimi Linear would be an example where NoPE worked well.