also, Llama3-70B is #1 on English-only, whut!!!!
RESEARCH
-
AI Generates Novel CRISPR Gene Editors Beyond Natural Origins
By
–
New A.I. technology generates CRISPR gene editors that do not come from nature:
-
Faster AI Inference Enables More Thinking Same Time
By
–
The model will output the same result, no matter the speed. What faster inference enables is more thinking in the same amount of time.
-
AI Model Learning from Inference Failures and Preferences
By
–
A few bits per pair, sure, but so nutritious — lessons not just from humans but from its own inference failures, categorically missing from pre-training. Preferences are some of the only bits a model gets on how to succeed at being itself, with its specific quirks and deficits.
-
Getting into AI: From GPT-2 fine-tuning to ChatGPT
By
–
This was probably around late 2020, 2021. Before that I was doing a bit of fine-tuning over GPT-2, but chargpt was when I really got into it.
-
Character-Level GPT Models Launch ML Training Journey
By
–
Dude…. big fan. Your character-level GPT models were how I initially got into training models
-
Explainable AI: Challenges Across Telecommunications and Research
By
–
Looking to gain a deeper understanding of Explainable AI? Experts from Eckerson Group, Orange and Georgia Tech dive into the challenges of AI explainability from telecommunications networks to academic research. https://
buff.ly/3vNcwMr -
RoPE Scaling Enables 16K Token Context Window Models
By
–
Yeah — simply multiplied rope * 2, trained on long data with a MSL of 16K.
-
LLaMA 3-based model deployment nearing completion
By
–
Very soon. This is one of the last things I have to get working well before we can push a LLaMA 3-based model live.
-
Llama 3 Announcement Highlights Data Quality’s Impact on AI Performance
By
–
Llama 3 announcement says the quality of both SFT and preference data have an “outsize influence” on performance. The first excerpt above alludes to an explanation for why preferences would help with reasoning. (This screenshot is the text just before the one above:)