And @calebfahlgren wrote a short article on how to replicate it using Hugging Face's SQL console https://
huggingface.co/blog/cfahlgren
1/the-beginners-guide-to-cleaning-a-dataset
… I had no idea it was an option and it's a lot faster than my method

→ View original post on X — @maximelabonne,