Hm, as far as I know, Copilot is not an imitation model though, right? Wasn't it pretrained on GitHub code repositories?
@rasbt
-
Open Source LLM Research Progress Motivated by GPT-4 Competition
By
–
On the flip side, there’s been a lot of progress on open source LLM research lately because of GPT-4. “Out of spite” is a strong motivator
-
Morning AI News Routine Before Work Starts
By
–
Haha that makes me feel better. Usually, I catch up to AI news (that I sent to my e reader on the previous day) with my first morning coffee at 5 am. And then I usually read ML / AI research papers or write, before the workday (aka meetings) starts at 8. AI is a commitment
-
Notable ML researchers must choose between podcasts or blocking
By
–
Haha true. Once you are notable enough as a machine learning / AI researcher you either go on the podcast or get blocked 😛
-
Imitation Models Mimic Style, Not Factuality of LLMs
By
–
The False Promise of Imitating Proprietary LLMs (
https://
arxiv.org/abs/2305.15717) Crowd workers rate imitation models (models trained on other models' outputs) highly. But it turns out that these imitation models only tend to mimic the style of the upstream LLMs not their factuality. -
Full Finetuning Not Always Necessary for Large Language Models
By
–
I don't think full finetuning is necessary, except as a baseline perhaps. Even without PEFT, it often doesn't make sense to tune all layers: (from my article https://
magazine.sebastianraschka.com/p/finetuning-l
arge-language-models
…) -
Ligand Discovery Projects Confuse Active Molecules with Drugs
By
–
Haha good point. 99% of computational ligand discovery project conflate ‘active molecule’ with ‘pharmaceutical drug’
-
Common buzzwords in technology and business sectors
By
–
buzzword check: AI Data science Disrupt Big data
-
Curating Interesting and Useful AI Research Papers Monthly
By
–
yeah, that sounds about right. I spend a lot of time each month combing through papers that are interesting but also useful and probably worthwhile E.g., "AI Research Highlights In 3 Sentences Or Less (May-June 2023)" https://
magazine.sebastianraschka.com/p/ai-research-
highlights-in-3-sentences-2a1
… -
Improved ArXiv Integration with Peer-Review Indicators
By
–
What might be more feasible in the meantime is to implement a better integration with journals & conferences so that arxiv papers get a more prominent "peer-reviewed" indicator once the paper passes said reviews. (Right now it's on the authors to update)