Sorry can you expand? Are you saying this might be due to the flash attention inside cuDNN?
CODE
-
Best Machine Learning Books: Essential Reading Guide
By
–
Best #Books for Machine Learning You Must Read! #BigData #Analytics #DataScience #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #CloudComputing #Serverless #DataScientist #Linux #Programming #Coding #100DaysofCode https://
geni.us/Best-A-I-Must-
Read
… -
Essential Data Science Books for ML and Analytics
By
–
Top Books You Must Read! #BigData #Analytics #DataScience #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Books #Programming #Coding #100DaysofCode https://
geni.us/15-DSci-Books -
Azure PowerShell Quick Start Guide for Cloud Computing
By
–
#Azure PowerShell Quick Start Guide! #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #Python #RStats #TensorFlow #JavaScript #ReactJS #CloudComputing #Serverless #DataScientist #Linux #Books #Programming #Coding #100DaysofCode https://
geni.us/Powershell-SG -
Popular Machine Learning Algorithms Explained Comprehensive Guide
By
–
Popular #MachineLearning #Algorithms Explained. #BigData #Analytics #DataScience #AI #IoT #IIoT #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Programming #Coding #100DaysofCode https://
geni.us/10-Popul-ML-Ex
plained
… -
100 Machine Learning Projects with Python Guide
By
–
100 #MachineLearning Projects with #Python. #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Programming #Coding #100DaysofCode https://
geni.us/100-ML-Project
s-Python
… -
FineWeb Dataset Wins Test of Time Award
By
–
Test of time award! https://
huggingface.co/spaces/Hugging
FaceFW/blogpost-fineweb-v1
… -
Exporting Model Weights for Faster Evaluation Infrastructure
By
–
Instead of building them out inside llm.c it might be faster to export the model weights into "common infra" and run evals with that. I don't have time to get around to it right away but made an Issue a few days ago for someone to potentially take a look.
-
Pretraining Datasets and Evaluation Metrics for Multilingual Models
By
–
Yes, definitely, my last tweet few seconds ago is also on this point. And many pretraining datasets also care about e.g. multilignual, code, math, etc., so it's not clear how those evals would be affected.