Great! Any timeline there? Waiting to really push hard on training LLaMA 3 till I can use long-context.
RESEARCH
-
Llama 3 8B Performance Comparable to Llama 2 70B Model
By
–
The model card has some more interesting info too: https://
github.com/meta-llama/lla
ma3/blob/main/MODEL_CARD.md
… Note that Llama 3 8B is actually somewhere in the territory of Llama 2 70B, depending on where you look. This might seem confusing at first but note that the former was trained for 15T tokens, while the -
Flow Engineering with CodiumAI and LangChain LangGraph Webinar
By
–
Flow Engineering with CodiumAI & LangChain/LangGraph New Webinar Alert – tomorrow at 9 AM PT! https://
us06web.zoom.us/webinar/regist
er/WN_fVikSl9eQv68b3ZUdQgwzA#/registration
… "Flow Engineering" is a term that has been gaining in popularity recently. The first time it was mentioned as term was in @CodiumAI paper on AlphaCodium, -
Llama 3 70B Now Available on Poe Platform
By
–
Llama 3 is now available on Poe! Llama 3 70B is now the most powerful open source model available, with state-of-the-art performance across industry benchmarks. It delivers new capabilities like enhanced reasoning, better code generation, and improved instruction following. (1/2)
-
Cognitive AI vs Traditional AI: Key Distinctions for Innovators
By
–
Unveiling the distinctions between Cognitive AI & traditional AI — crucial for innovators! Embrace the future of technology with insights from @ingliguori and level up with 'The Digital Edge' https://
bit.ly/3u4pILl #ArtificialIntelligence #CognitiveAI #TechTrends -
Top AI Experts in NYC Worth Interviewing
By
–
Who are your favorite AI people in NYC that you want to see interviews with?
-
SQuAD Explorer: Benchmark for Machine Reading Comprehension
By
–
Oh yeah, this looks good https://
rajpurkar.github.io/SQuAD-explorer/ -
Chinchilla Scaling Laws: Compute Optimality vs Convergence Point
By
–
no. people misunderstand chinchilla.
chinchilla doesn't tell you the point of convergence.
it tells you the point of compute optimality.
if all you care about is perplexity, for every FLOPs compute budget, how big model on how many tokens should you train?
for reasons not fully -
Meta Releases Llama 3 Models in 8B and 70B Variants
By
–
Congrats to @AIatMeta on Llama 3 release!! https://
ai.meta.com/blog/meta-llam
a-3/
…
Notes: Releasing 8B and 70B (both base and finetuned) models, strong-performing in their model class (but we'll see when the rankings come in @ @lmsysorg :))
400B is still training, but already encroaching -
Compute Optimal Inference Setups Explained
By
–
well, it's compute optimal for some inference setups, I think that's the idea; it's not optimal for training though