Even fine tuning regular lora with adafactor would do for now though. With activation checkpointing that’ll handle a reasonable size model
RESEARCH
-
QLoRA Fine-Tuning for Single Box Decoder Applications
By
–
I think you really need a single box decoder only fine tuning task, since that’s a really common application. E.g qlora fine tune mistral on flan v2
-
AI Learning Loss Predictions: Initial Estimates Proved Conservative
By
–
I non-ironically had 10% on “no detectable learning loss.” Glad I was wrong, I suppose, in a limited sense.
-
Ilya Sutskever presents GPT-2, foundation of ChatGPT
By
–
Ilya Sutskever @ilyasut presenting GPT-2 back in March 2019, the core of ChatGPT. pic.twitter.com/9I6IUbq3JA
— Reza Zadeh (@Reza_Zadeh) 19 novembre 2023Ilya Sutskever @ilyasut presenting GPT-2 back in March 2019, the core of ChatGPT.
-
Pragmatic Accelerationism: A Philosophy for AI Development
By
–
Yes, this sounds like you. I consider myself a pragmatic accelerationist.
-
OpenAI and Sam Altman: Key Figures in AI Development
By
–
Yeah, let’s talk about OpenAI and Sam Altman now.
-
Personality Traits Shape Our AI Future Stances
By
–
Our stances as accelerationists, alarmists, or skeptics are deeply rooted in our personalities, extending beyond mere rational logic. My own personality traits play a crucial role in shaping my perception of the world and the future. A person's openness to change and
-
Fine-tuning Small AI Models for Efficient and Ethical Deployment
By
–
The conversation highlights the value of fine-tuning small AI models for specific tasks to reduce power and computational needs, emphasizing responsible, ethical scaling in AI deployment and development. We discuss tons of other fascinating AI-related topics 🙂
-
AI Expert Interview: Future Trends and Sector Impacts
By
–
Explore the future of AI and its sector impacts through insights from an AI expert in this interview. Watch the full interview on
-
AI Hallucinations: Solutions Through Human Feedback and Uncertainty
By
–
The interview covers AI's risk of outputting false information, which we usually refer to as "hallucinations". She highlights solutions like human-informed reinforcement learning and systems seeking clarity during uncertainties.