theoretically, the implications of this seem big. we call it The Strong Platonic Representation Hypothesis: models of a certain scale learn representations that are so similar that we can learn to translate between them, using *no* paired data (just our version of CycleGAN)
@jxmnop
-
Aligning AI Model Representations Through Structure Matching
By
–
we take things a step further. if models E1 and E2 are learning 'similar' representations, what if we were able to actually align them? and can we do this with just random samples from E1 and E2, by matching their structure? we take inspiration from 2017 GAN papers that
-
GAN Training for Cross-Model Embedding Alignment
By
–
so yes, we're using a GAN. adversarial loss (to align representations) and cycle consistency loss (to make sure we align the *right* representations) and it works. here's embeddings from GTR (a T5-based model) and GTE (a BERT-based model), after training our GAN for 50 epochs:
-
All Embedding Models Learn The Same Thing
By
–
excited to finally share on arxiv what we've known for a while now:
— dr. jack morris (@jxmnop) 21 mai 2025
All Embedding Models Learn The Same Thing
embeddings from different models are SO similar that we can map between them based on structure alone. without *any* paired data
feels like magic, but it's real:🧵 https://t.co/Cwj1LytGosexcited to finally share on arxiv what we've known for a while now: All Embedding Models Learn The Same Thing embeddings from different models are SO similar that we can map between them based on structure alone. without *any* paired data feels like magic, but it's real:
-
Scale Convergence: Different AI Models Learn Identical Representations
By
–
a lot of past research (relative representations, The Platonic Representation Hypothesis, comparison metrics like CCA, SVCCA, …) has asserted that once they reach a certain scale, different models learn the same thing this has been shown using various metrics of comparison
-
AI Industry Racing to Ship the Obvious Features
By
–
much of AI these days is a race to ship the obvious
-
French LLM startup’s multilingual model race strategy
By
–
heard a funny story about a friend who worked at a French LLM startup a couple yrs ago > their plan. to be the first to market with a certain type of multilingual model
> early in year: incorporate. start building
> hired an awesome team by march
> scraped / acquired all the -
Mistral AI model status and recent developments unclear
By
–
i guess there was mistral for a while, idk what happened there
-
Model Weights vs API Access: Information Extraction and Security
By
–
that’s a good point. with an API you can make lots of little updates, which providers definitely do what i meant was that people can learn a lot from the weights – info about the training process and tokenizer and data and architecture. stuff you can’t get from an API
-
Meta’s Open-Weight Models: Bold Move vs Industry Caution
By
–
people that don't know love to criticize Meta on twitter, since it's guaranteed engagement but you have to realize that releasing open weights puts you in a vulnerable position. it's scary, and hard. that's why no one else is doing it google's gemma is open, but small. AI2's