Oh yes, for sure. Everyone is (still) working on transformers atm.
@rasbt
-
GPT-4 limitation: inability to create and annotate figures
By
–
The bottleneck is that GPT-4 cannot create or annotate figures. And I honestly can't read surveys or reviews without figures 😛
-
Recent LLM Research Highlights: Llama 2, ReLoRA, AlpaGasus
By
–
New base models with Llama 2, pretraining LLMs with ReLoRA, training with less data via AlpaGasus, and many more … There's been lots of interesting research in the last month! I summarized the research highlights in three sentences each:
-
NCD vs Cosine Distance for Vector Comparison
By
–
Nice one thanks for sharing. Looks like a really nice write-up. I remember running the NCD on count vectors, but then I thought cosine distance is even simpler. Intuitively, I'd say they should capture about the same thing.
-
Tinygrad Deep Learning Framework AMD Support Discussion
By
–
Oh interesting, I know tinygrad (the DL framework)! Wait, so it does support AMD or is it in the works? Also, is this going to be without ROCm then?
-
AMD AI Tools Company Startup and ROCm Usability Status
By
–
Wasn't there someone who recently started a company around building AI tools specifically for AMD … and then gave up after like a few weeks? How useable is ROCm at this point? (I know PyTorch has support for it, but it is still experimental)
-
Excitement for Efficient LLM Fine-tuning Research Methods
By
–
I am really excited for the research community to develop (more) efficient methods for finetuning LLMs.
And I hope you find this competition as useful and exciting as I do! Link to the competition here: -
NeurIPS 2023 LLM Efficiency Challenge Starter Guide Released
By
–
The NeurIPS 2023 LLM Efficiency Challenge is a super exciting opportunity for developing & benchmarking new research directions for param-efficient LLMs. If you are looking for something to tinker with this weekend, I just put together a Starter Guide: https://
lightning.ai/pages/communit
y/tutorial/neurips2023-llm-efficiency-guide/
… -
Question about paper submission phrasing for real conferences
By
–
would we say "we submitted a paper to the ICML" if it was a real conference? 😛
-
arXiv Papers Compared to GitHub Code Repositories
By
–
“The paper is on the arXiv” would be like “the code is on the GitHub”