Sunday small guessing puzzle Let's say I have 3 tokenizers:
– llama2: 32k vocab
– falcon: 65k vocab
– GPT4: 100k vocab I take ~2M random documents from the web (let’s say 10 random parquet files from RefinedWeb from https://
huggingface.co/datasets/tiiua
e/falcon-refinedweb
… roughly 1B tokens). I tokenize them
MACHINE LEARNING
-
Tokenizer vocabulary size comparison across language models
By
–
-
Edge Detection Limitations in Machine Learning Line Drawing
By
–
Why Edge Detection Doesn’t Explain Line Drawing https://
bit.ly/3quKkLn #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Transformers Can Learn Hierarchical Structures Through Parse Trees
By
–
Why do you say we haven’t solved “hierarchical structure”? Transformers can learn parse trees & bracketing, each of which is full generalizable to arbitrary hierarchies. I think we totally machine learn hierarchical structures now.
-
Researchers Discovered ChatGPT Vulnerabilities and Future AI Implications
By
–
How researchers broke ChatGPT and what it could mean for future AI development https://
zd.net/3YrH7J1 #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Making Scientific Data Accessible and Discoverable for AI
By
–
How to make your scientific data accessible, discoverable and useful https://
go.nature.com/44WvGvx #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Satya Nadella’s Intelligence Is Not Artificial
By
–
Satya Nadella’s Intelligence Is Not Artificial – Freakonomics https://
bit.ly/3QvoIcG #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Large Language Models as Optimizers: Yang et al Research
By
–
Large Language Models as Optimizers Yang et al.: https://
arxiv.org/abs/2309.03409 #ArtificialIntelligence #DeepLearning #LargeLanguageModels -
What Changed in AI Since 2019
By
–
Has anything changed since 2019, @cra
? I'm writing on the topic now…. -
PhD Deep Learning: Unnecessary in Mainstream AI Era
By
–
i think you're saying — reject the notion of a PhD or a research Masters in hot deep-learning topics because AI is now so mainstream and so well-capitalized and so accessible that you can do great work on discord or in an industrial lab and publish and you win at life.
That