10B tokens of FineWeb! Ilya said WebText was 40B tokens (
https://
youtube.com/watch?v=13CZPW
mke6A&t=3645s
… – for gpt2 1.5b) what accounts for the improved loss/accuracy that you got over GPT2 – have we improved our dataset filtering? were there smarter hparam choices made here? any ballpark attributions
MACHINE LEARNING
-
FineWeb Dataset: Improvements Over GPT-2 Training Data
By
–
-
Start Latency Issues with Dataset Tokenization in Python
By
–
TIL, will look into! The thing that makes this a bit complicated right now is the start latency. What bloats up the setup time right now is the dataset and its tokenization, which is all done in Python right now. Installing huggingface datasets, downloading FineWeb 10B and
-
Training Loss Noise and Validation Loss Smoothing Explained
By
–
Training loss is evaluated over the batch, i.e. 0.5M tokens. It's noisy but this is expected, you could be iterating through easy or hard documents in the training data. The validation loss is averaged over 20 batches of 0.5M tokens (this is a hyperparameter), so it is smoother.
-
Conservative hyperparameters with potential for 10X speedup
By
–
Agree!! I'm using very conservative settings for a lot of the hyperparameters (following GPT-3 paper when possible) and haven't tried to speed this up at all yet, but I expect a 10X multiplier here should be possible.
-
McDonald’s Uses ML for AI-Driven Restaurant Site Selection
By
–
Restaurant expansion is a key initiative for the data team at McDonald’s, and #AI-driven decisions pave the way. Join this #DataAISummit session to explore #ML techniques used to drive the strategy for new site selection https://
dbricks.co/3IMHmaT -
ECCV 2024 Workshop on Audio-Visual Generation and Learning
By
–
Initial CfP advertisement for the ECCV 2024 workshop "AVGenL: Audio-Visual Generation and Learning". It will cover a wide range of topics about audio-visual generation and learning. Paper submission deadline is 15 Jul 2024. #ECCV2024 #ECCV @eccvconf
-
Fine-tuning Faster R-CNN for Sea Rescue Detection
By
–
Fine-tuning Faster R-CNN for Sea Rescue 🌊https://t.co/U1G0q7YrSk
— Satya Mallick (@LearnOpenCV) 28 mai 2024
Our research enhances Faster R-CNN to detect people in distress using the SeaDroneSee dataset. By preprocessing images into patches, we significantly improve detection accuracy.
Dive into our findings and see… pic.twitter.com/YQ5KOfvUClFine-tuning Faster R-CNN for Sea Rescue https://
learnopencv.com/fine-tuning-fa
ster-r-cnn/
… Our research enhances Faster R-CNN to detect people in distress using the SeaDroneSee dataset. By preprocessing images into patches, we significantly improve detection accuracy. Dive into our findings and see -
Voyager SDK enables YOLOv5 L PPE detection deployment
By
–
Explore the ease of our Voyager SDK in this video with @fogsphere. In just minutes, they effortlessly deployed the #Yolov5 L model for PPE detection with our AI acceleration technology. Don't miss your chance to innovate with our Metis AI platform. https://t.co/XYy5Ncb30k pic.twitter.com/i8L2DqNkyb
— Axelera AI (@AxeleraAI) 28 mai 2024Explore the ease of our Voyager SDK in this video with @fogsphere
. In just minutes, they effortlessly deployed the #Yolov5 L model for PPE detection with our AI acceleration technology. Don't miss your chance to innovate with our Metis AI platform. https://
tinyurl.com/244md3rf -
Open-Source Self-Driving Model Gains Community Traction
By
–
very cool! could be the first very good model of its kind on http://
hf.co/models for the open-source community to focus more on self-driving systems. Happy to help amplify! -
Graph-Based RAG Paper Releases Benchmarks and Open Source Code
By
–
ooooh a graph based RAG paper w benchmarks and open source code