I'm such a good businessman I forgot the link. Here you go: https://
amazon.com/LLM-Engineers-
Handbook-engineering-production/dp/1836200072/
… Thanks @sommelierhela2
! <3
@maximelabonne
-
LLM Engineers Handbook: Production Engineering Guide
By
–
-
LLM Engineer’s Handbook: Top Gift for Neural Networks
By
–
Christmas gift ideas: the LLM Engineer's Handbook is a safe bet, currently #1 in neural networks Turn your partner into an LLM Engineer with this one weird trick!
-
AI Evaluation Methodology: Beyond Heuristic Approaches
By
–
It doesn't really measure what I mentioned (throughput, CoT, etc.) but yes, it's better overall. It might still be fundamentally flawed because it tries to fix this issue with heuristics instead of a more accurate evaluation.
-
LLMs Better Than Humans at Content Evaluation Tasks
By
–
Yes, I think it's harder for humans to evaluate, but probably not for LLMs
-
Chatbot Arena Evaluation Flaw: Messages vs Problem-Solving
By
–
I think the main issue with the chatbot arena is that it evaluates messages instead of conversations. Users interact with LLMs to solve specific problems. Ideally, evaluations should capture how well this problem is solved. → When o1 takes 2 minutes to give me an answer
-
Answer-Based Evals Limitations and Conversation-Based Comparisons
By
–
Agreed, I don't like this behavior either. Imo, this is a limitation of our answer-based evals, where answers with more information are preferred over shorter ones. Conversation-based comparisons might prevent this because you'd judge the entire experience.
-
Best Practitioners Fine-Tune AI Models on PDF Documents
By
–
This is also my experience. The best of the best are the people who want to fine-tune on PDFs.
-
Prometheus 2 Model Merging with LazyMergekit Technique
By
–
True, this is even older than this paper. Several authors pinged me about it like Prometheus 2 (they used LazyMergekit haha): https://
arxiv.org/abs/2405.01535 -
Merging Smol-Talk and Orca-AgentInstruct Models
By
–
Secondary question: what if you merge the model trained on Smol-Talk and the one trained on Orca-AgentInstruct-1M?
-
Combining Smol-Talk with Orca-AgentInstruct Models
By
–
Another excellent release from @huggingface My main question is what if you combine Smol-Talk with Orca-AgentInstruct-1M? I'd bet it works, they don't seem super redundant.