I asked @wzhao_nlp who is the expert on this, she think it could be a fine-tuned classification model, and a small classifier would work really well for this I think
@jxmnop
-
BM25 Baseline and Embeddings Beyond Retrieval Applications
By
–
right, all these papers use BM25 as a baseline and it's super good especially in weird domains but embeddings are used for more than just retrieval
-
Google’s GTR Paper: Embedding Model Scaling Results
By
–
I know this result from google's GTR paper (
https://
arxiv.org/pdf/2112.07899
.pdf
…) they scale their embedding model and it gets a little better, but not much -
Why Aren’t We Training Massive Embedders?
By
–
why aren't we training massive embedders? some guesses:
– contrastive loss isn't the right loss function for embeddings
– not enough good paired data
– unclear what the use case is (retrieval? clustering? classification?)
– no principled scaling laws
– diminishing returns -
GTR: Google’s Neural Retrieval Model for Information Retrieval
By
–
barely though this is from GTR, the Google retrieval model (
https://
arxiv.org/pdf/2112.07899
.pdf
…) -
Concurrent Research on AI Paper Released Earlier
By
–
actually @ZhongRuiqi was working on this at the same time, and released his paper a few days earlier, I just checked 🙂 https://
arxiv.org/abs/2104.04670 -
No Free Lunch in Convex Optimization: Momentum Limitations
By
–
lol I received several messages telling me to learn more about optimization so I might have to do that and think about it more my thought process was that there's no free lunch in convex optimization so even if eg momentum works in some cases, it doesn't work in others, which is
-
Exploring AI Career Path: A Serious Consideration
By
–
few enough for me to seriously consider this career path
-
PhD dropout considers piano tuning career over computer science
By
–
thinking of quitting my computer science phd program to become a professional piano tuner. many apparent benefits
– get to work with hands (touch grass every day)
– solve interesting problems (tuning pianos)
– make more money (many pianos, few tuners, basic supply and demand)