Yes, any finetuning method is allowed, afaik. Technically, you could also just further (pre)train the model. Anything that runs <24 h on open source code and openly available data.
@rasbt
-
Lit-GPT: Unified Codebase for Decoder Analysis and Comparison
By
–
It’s kind of hard to analyze across repos due to implementation differences and details along the data loading and finetuning pipelines. That’s where I’d say Lit-GPT is useful because it makes the set of relevant decoders available in the same unified code base
-
Model Documentation Gaps and Empirical Analysis Importance
By
–
Not that I am aware of. On top of that several models don’t even have thorough papers themselves because people are currently rushing them out. I guess the best way is really some empirical analysis coupled with some knowledge bits like falcon and llama2 use multiquery attention
-
Mastering LLM Basics as Foundation for Innovation
By
–
I don't disagree, but I think that in order to innovate it can also be helpful to have the basics down. I.e., implementing and running state-of-the-art LLMs is kind of part of the baseline exercise before going further from there.
-
NeurIPS LLM Efficiency Challenge: Train Model in One Day
By
–
Trying to develop the next huge LLM is fun, but how about a side project with a more reasonable scope like the 1 LLM + 1 GPU + 1 Day NeurIPS efficiency challenge: https://
llm-efficiency-challenge.github.io Below a list of the approved models (PS: Lit-GPT was chosen as the official starter kit ) -
Joining DS/ML Book Club Discussion on Machine Learning and AI
By
–
Looking forward to joining @sophiamyang 's
DS/ML book club tomorrow over lunch (1 pm CT) to talk & discuss Machine Learning Q and AI (
https://
leanpub.com/machine-learni
ng-q-and-ai/
…)! This book is a bit more advanced than my previous ones, covering concepts such as – Multi-GPU training paradigms. -
Tie-breaking fix improves nearest neighbor classification accuracy to 71.91%
By
–
Small update: In the orig code example in the paper, tie breaking was based on the lowest class label index, i.e., it was biased towards class 0. When I fixed the tie-breaking to select the closest neighbor, I could get a 70.05% -> 71.91% improvement (
https://
github.com/rasbt/nn_plus_
gzip
…). -
Method’s Limitations Compared to Large Language Models
By
–
Oh yes, of course! The method is nice, but yeah, it would be a bit weird if it could outperform LLMs on things that LLMs are good at.
-
Machine Learning Class Coverage of KNN Lecture Notes
By
–
I usually cover these in my machine learning classes https://
sebastianraschka.com/pdf/lecture-no
tes/stat451fs21/02-knn__slides.pdf
… -
Background execution and minimal pseudocode implementation approach
By
–
I let it run in the background, I think it was a day or so. But yeah, you can be much smarter about the implementation, I just kept it minimal, implementing the exact pseudocode they had in the paper