super cool! I wonder if it might work better if you work in the space of a pretrained embedding. care to share some insight on why this doesn't work so well initially?
@jxmnop
-
ArXiv Paper Withdrawal Raises Questions About Research Standards
By
–
that's not what happens in this paper (also this is the first thing I've ever seen to be withdrawn from ArXiv, lol)
-
Embedding to Text Reconstruction: Three-Step Technical Approach
By
–
– step 1 can be implemented with any off-the-shelf embedder
– step 3 is solved with vec2text (
http://
github.com/jxmorris12/vec
2text
…)
– step 2 doesn't exist yet – some tasks might be solvable even with a simple embedding-to-embedding linear mapping if you're interested in trying this, DM me 🙂 -
Text Diffusion in Embedding Space for Sequence Tasks
By
–
cool research idea for someone: text diffusion in embedding space solve any sequence-to-sequence task in three steps:
1. embed source sentence text
2. build diffusion model that maps input text embedding to target text embedding
3. invert to produce target text -
Unknown Researcher’s Breakthrough at OpenAI Challenges Academic Expectations
By
–
now seems as good a time as ever to remind people that the biggest breakthroughs at OpenAI came from a previously unknown researcher with a bachelors degree from olin college of engineering
-
Scaling Alone Won’t Achieve AGI Says Researcher
By
–
this is a very good point, I guess this tweet comes with the association that I really don’t think scaling will get us to AGI
-
GPT Development Credits: Recognizing Key Contributors Beyond Papers
By
–
why! He’s a great communicator and thinker, I really admire him! But he wasn’t there when they built GPT-{1,2,3}, he isn’t on any of the papers
-
Twitter followers don’t predict long-term tech influence
By
–
well, you left out the “as of now” I think my main point is that Twitter followers doesn’t really correlate with influence on the long-term arc of technological progress
-
Fine-tuned Classification Model for AI Applications
By
–
I asked @wzhao_nlp who is the expert on this, she think it could be a fine-tuned classification model, and a small classifier would work really well for this I think
-
BM25 Baseline and Embeddings Beyond Retrieval Applications
By
–
right, all these papers use BM25 as a baseline and it's super good especially in weird domains but embeddings are used for more than just retrieval