a tiny summary of my research projects of 2023: Text Embeddings Reveal (Almost) As Much As Text EMNLP 2023, http://
arxiv.org/abs/2310.06816 Language Model Inversion ICLR 2024, http://
arxiv.org/abs/2311.13647 TreePrompting EMNLP 2023, http://
arxiv.org/abs/2310.14034
@jxmnop
-
Research Advances in Text Embeddings Language Models and Prompting
By
–
-
Acknowledging Key Contributor to AI Model Development
By
–
finally just want to give a public shoutout to @zach_nussbaum for all the hard work he put into making this model great. it wasn't a super clear path to this level of performance and it's inspiring to see people stick with projects through that. I certainly owe him a drink
-
Nomic Solves Supervised Embedding Space Interpolation Research Problem
By
–
also the embedding-space interpolation in the GIF is very new and incredibly cool; Nomic solved the supervised version of the research problem I tweeted about here:
-
Nomic Embed: Open Text Embedding Model and Dataset Released
By
–
if you're doing research on text embeddings, you know that there are lots of tricks required to train a good model, and no open datasets.
— dr. jack morris (@jxmnop) 1 février 2024
we trained Nomic Embed, a great text embedding model, and actually released the data! certainly will make my research easier – check it out! https://t.co/akhYHzjKbBif you're doing research on text embeddings, you know that there are lots of tricks required to train a good model, and no open datasets. we trained Nomic Embed, a great text embedding model, and actually released the data! certainly will make my research easier – check it out!
-
Carbon Dating ML Papers by Open-Source Language Models
By
–
carbon dating ML papers by which open-source LM they use GPT-2 → the before times
GPT-J → summer 2022
LLAMA-1 → spring 2023
LLAMA-2 → summer 2023
mistral 7b → fall 2023
mixtral 8x7b → the current era -
New Embedding Model Release Announced for Tomorrow
By
–
new embedding model dropping tomorrow…… stay tuned
-
Security versus Information Theory: Different Language, Same Concept
By
–
i think it just depends on the language of your field; security people call this an "attack"; information theory people might just call this "decoding"
-
Multilingual Text Embedding Inversion Attacks on Language Models
By
–
just read a cool follow-up to vec2text: "Text Embedding Inversion Attacks on Multilingual Language Models" these folks extend embedding inversion to the *multilingual* setting, where we might not know the language of the encoded text ahead of time they add a
-
Balancing Personal Goals Against Community Impact in Tech
By
–
yeah like balancing risk and reward/impact, whether to optimize for your personal goals or those of a wider community or what the difference there is, how to navigate the market of ideas, when to know how to quit and move on to something else vs blindly keep going
-
Five Deep Topics About AI, Research and Ethics
By
–
What are five topics you can talk about for 30 minutes with zero prep 1. information content of text embeddings
2. architectural mysteries of transformer language models
3. the meta-research process (how & why to pick problems)
4. whether the NBA is rigged
5. shrimp suffering