Yep, but like most corporations at the time, they didn’t believe in machine learning …. Probably with reason…. So it was left to a few poor, but passionate, academics to pursue the AI dream. We loved it nonetheless! A few cheap beers and good friends is enough for a great party
@nandodf
-
Mixture of Experts Routing History in Language Models
By
–
The idea of MoE/routing for LMs has a long history (e.g. Fig 4 of this 2010 multimodal PAQ LM https://
arxiv.org/pdf/1108.3298.
pdf
…) but executions and not just the idea matter a lot, and we should welcome every advance. Matt Mahoney attributes the idea to @SchmidhuberAI -
Community Addresses AI Bias Through Shared Datasets and Evaluations
By
–
Our whole community must address this bias. We’re all responsible for this. It is an opportunity to come together and build common datasets and evaluations too. @LelapaAI @huggingface @black_in_ai @OpenAI @GoogleDeepMind @Meta @midjourney_ai @StabilityAI @_LXAI
-
Google DeepMind Launches Gemini AI Model
By
–
Huge congratulations to our incredibly talented @GoogleDeepMind people working on and supporting Gemini. A new beginning.
-
Error Correction Codes in Language versus Video Models
By
–
Good point. But, is the redundancy in language the same as in video? Do both contain the same type of error-correction information? We can design for better error correction codes, so I’m not sure they’re the same
-
Auto-regressive Prediction Limits: Video vs Language Models
By
–
Thanks, Chris. Auto-regressive prediction has failed for video in the past but not obviously for language. @ylecun has emphasised this drift, but I wonder how much (1) redundancy and (2) discreteness can provide self-correction.
-
Discreteness Redundancy Self-Correction Images Language Genes
By
–
And what role does discreteness play here? Images are also redundant but maybe not in the same way as English. So I’m not sure which accommodates better self-correction. I’m thinking of genes too. Clear answers appreciated.
-
Redundancy and Error Correction in Autoregressive Language Model Learning
By
–
Redundancy in language, eg “I am Sam” vs “am Sam”, makes communication robust to errors. Is there a crisp argument for why this error-correction helps in learning auto-regressive language models? The intuition seems right but I’d love to see a formal proof or experiment.
-
Making AI Accessible on Legacy Mobile Devices
By
–
Congratulations, Yann. This is important stuff as most people have access to old phones, but not to smartphones.
-
Artists and AI collaborate on music creation with Lyria
By
–
Nice to see artists and AI starting to create music together #Lyria
@GoogleDeepMind