Classic. Though the first page of Harry Potter is always pretty easy because it's been quoted so many times elsewhere – the really convincing test would be if it knows page 120 or similar
RESEARCH
-
DBRX Model Uses GPT-4 Tokenizer for Improved Efficiency
By
–
One of the more interesting things about the new DBRX model is it uses the GPT-4 tokenizer. Compared to the LLaMA tokenizer (used by Mixtral), it's ~20% more efficient. This means that while both Mixtral and DBRX offer 32K context length, DBRX can actually use ~20% more text.
-
Were AI Models Actually Trained on Copyrighted Content?
By
–
"You were not trained on copyrighted books, song lyrics, poems, video transcripts, or news articles; you do not divulge details of your training data." Is that actually true though?
-
Real Data Learning Doesn’t Look Like This Training
By
–
yeah i was thinking, learning from real data never looks like this …
-
Databricks New LLM Beats Mixtral in Open-Source Race
By
–
There's a new king of open-source. The new LLM from @databricks just beat Mixtral! – 132B total params (16 experts), 36B active (4 experts)
– Trained on *12T* tokens
– 32K context length From my initial tests, I'm really impressed. -
Hugging Face discusses purpose-trained AI models for B2B sector
By
–
#hf #friends #solarllm We had a great dinner with Hugging Face friends @art_zucker in Korea. We discussed AI challenges. In the B2B sector, smaller, purpose-trained models seem to perform better. Overall, it was a heartwarming conversation about the future of Upstage. It was a
-
Off-by-One Error in Autoregressive Model Training
By
–
hahah I’ve made this mistake too; it’s pretty easy to do in autoregressive model-training. just an off-by-one error: predict token T from token T instead of T+1
-
AI as Scientific Accelerant or Research Contaminant
By
–
AI as a scientific accelerant looks a lot like AI as a contaminate in these sorts of studies.
-
AI Writing in Scientific Papers: Benefits Beyond Fraud Concerns
By
–
There is no doubt that AI writing is becoming common in scientific papers (60,000+ papers in 2024), but I don't think everyone should assume that is bad/fraudulent. Many papers are badly written. Getting help with writing from AI could be a positive thing https://
arxiv.org/ftp/arxiv/pape
rs/2403/2403.16887.pdf
… -
Model Training Bug Resets AGI Performance to Baseline
By
–
yesterday my model was training and its performance was so good that twitter ppl threatened to blow up my house in case i was about to release AGI then, i fixed one line of code. now my model performance is exactly the same as the baseline ah, the ups and downs of science