(or mmlu is leaking into t he data)
LLMS
-
llm.c Optimization: Matching PyTorch Performance After Bug Fix
By
–
Highly amusing update, ~18 hours later: llm.c is now down to 26.2ms/iteration, exactly matching PyTorch (tf32 forward pass). We discovered a bug where we incorrectly called cuBLAS in fp32 mathmode . And ademeure contributed a more optimized softmax kernel for very long rows
-
End to End Latency vs Opus Weaviate Embeddings
By
–
whats end to end latency? vs just opus + weaviate + embeddings
-
Using Large Language Models: A Year-Long Series Guide
By
–
A year later here's my series of posts about this https://
simonwillison.net/series/using-l
lms/
… -
DBRX: New Open-Source LLM Outperforms Established Models
By
–
#DBRX is a new general-purpose LLM that outperforms all established open-source models on standard benchmarks!
— Databricks (@databricks) 13 avril 2024
DBRX is incredibly efficient thanks to its fine-grained Mixture-of-Experts architecture. Learn how you can get started👇https://t.co/vtG4r2m5hY pic.twitter.com/9o5agDAJBk#DBRX is a new general-purpose LLM that outperforms all established open-source models on standard benchmarks! DBRX is incredibly efficient thanks to its fine-grained Mixture-of-Experts architecture. Learn how you can get started https://
dbricks.co/49g4ZDt -
Tokenizer Training Data: Understanding Token Presence Significance
By
–
If uberinternal WAS a token that tells you that it was one of the top ~30,000 character sequences present in the text they used to build the tokenizer – which is a different corpus from the training set used to train the model uberinternal not being a token doesn't tell you much
-
AI Agents Limited by GPT-4 Brain but Show Huge Potential
By
–
I see a lot of debate on this site, so two things about the current state of AI agents in the real world
1) Agents remain limited by GPT-4 brains, it is early
2) Both academic testing & my own subjective experience (
https://
oneusefulthing.org/p/which-ai-sho
uld-i-use-superpowers
…) suggest agents have huge potential -
Mixtral Training Challenges: Memory Issues and Model Recovery
By
–
Training Bixtral, which is expensive + slow so mistakes are costly. The run finally finished, but then we got a sigkill -9 error (I believe it's a memory issue). I thought I lost the model, but I went into the output directory anyway, and luckily, it was there.
