Hm, I am pretty sure that LoRA, Adapters, etc all came from open source research communities. Also many (post)quantization methods.
@rasbt
-
Speculation on Model Distillation and Gemma-Gemini Similarities
By
–
Good catch. Yes, maybe that's distilled down from a larger model they didn't open source. In that context, I am also curious how similar Gemma and Gemini models are.
-
New Model Shows Strong Performance Against Llama 3
By
–
I only queried it for a bit and it seems to do well compared to llama 3
-
Gemma 3 Technical Report and Memory Usage Analysis Script
By
–
Here's a link to the Gemma 3 technical report: https://
storage.googleapis.com/deepmind-media
/gemma/Gemma3Report.pdf
… I shared the script to replicate the memory usage vs context length analysis here: https://
gist.github.com/rasbt/b7980a2a
33e33847f08b1543f0627950#file-llama-vs-gemma-py
… (not very pretty but it should work) -
Gemma 3 Analysis: Large Vocabulary Size and Model Insights
By
–
Just read through the Gemma 3 report and toyed around with the models a bit, and there are a bunch of interesting tidbits: 1. Vocab size. They again use a very large vocab: 262k token (in contrast, Llama 3 has ~1/2 the vocab size), which should make the model more friendly for
-
Uncertainty about Gemma and Gemini model similarities
By
–
sure, I don't know though how much Gemma and the flavor of Gemini they are using there have in common though.
-
Local Model Comparison Issues with Llama and Alternatives
By
–
When trying them out locally, they usually compared favorably to Llama etc. What were the issues you were having?
-
Gemma Models: Underestimated LLMs with New Technical Report Released
By
–
Great news! Gemma models are one of the most underestimated LLMs imho. The technical report is available here: https://
storage.googleapis.com/deepmind-media
/gemma/Gemma3Report.pdf
… -
Congratulations on new model release and request for research paper
By
–
Big congrats! The Gemma 2 model was awesome, and I look forward to trying this one! Btw is there a paper forthcoming similar to Gemma 2? Would love to read about it in more detail!
-
Using Meta pretrained weights without training or image tokens
By
–
Hm, did you use the pretrained weights from Meta? Does the model work fine if you don't do any training and don't pass any image tokens?