Hard to tell given that comparisons with gpt5-codex and gemini-2.5-pro etc are missing
@rasbt
-
Building and Pre-training Neural Network Architecture from Scratch
By
–
We build the architecture from scratch and pre-train it on a small dataset. But we keep the pre-training short to keep it accessible. Then, we load pre-trained weights and further fine-tune it.
-
Stable Diffusion: A Retrospective on Naming and Legacy
By
–
In hindsight, Stable Diffusion was actually a good one. Sure, didn't end well, but the name itself wasn't a bad choice.
-
Major Labs Avoid Fine-tuning Competitors’ Models Due to Company Policy
By
–
Fair. I think a lot of that is because the major labs are commercial labs, and they can’t (due to company policy)/don’t like to finetune competitors’ models
-
Nvidia Nemotron’s Superior Performance Influences Llama Model Improvements
By
–
Historically Nvidia’s Llama Nemotron models always outperformed the official Llama variants, and I am sure the llama team took a look at what they did to improve their own pipeline. I am sure they benefited from this, and this wouldn’t have been possible without open source.
-
Community Fine-tuning Benefits for Model Developers
By
–
I think finetuned models can also be a benefit for the developers. Ie developers may release a base model, and the community applies continued pretraining and finetuning to make a Math, Code, Instruct, Reasoning variant etc so the developers don’t have to.
-
LoRA and Quantization Methods Origins in Open Source Research
By
–
Hm, I am pretty sure that LoRA, Adapters, etc all came from open source research communities. Also many (post)quantization methods.
-
Speculation on Model Distillation and Gemma-Gemini Similarities
By
–
Good catch. Yes, maybe that's distilled down from a larger model they didn't open source. In that context, I am also curious how similar Gemma and Gemini models are.
-
New Model Shows Strong Performance Against Llama 3
By
–
I only queried it for a bit and it seems to do well compared to llama 3
-
Gemma 3 Technical Report and Memory Usage Analysis Script
By
–
Here's a link to the Gemma 3 technical report: https://
storage.googleapis.com/deepmind-media
/gemma/Gemma3Report.pdf
… I shared the script to replicate the memory usage vs context length analysis here: https://
gist.github.com/rasbt/b7980a2a
33e33847f08b1543f0627950#file-llama-vs-gemma-py
… (not very pretty but it should work)