High alpha trick for fine-tuning: Make your system prompts in your dataset really great. It'll help the model learn to do your task much faster, with less data. If you have lots of data, you can ignore this, but at small dataset sizes, this changes everything.
@mattshumer_
-
Strong AI Benchmarks Achievement Recognized
By
–
Benchmarks look really good… congrats @erhartford
! -
HyperWrite Helps Users Generate Ideas for Content Creation
By
–
It seems like it’d be a great fit! Many of our users use HyperWrite to help them with ideas, and then they put the final pieces together themselves.
-
Data Is Everything in AI Development
By
–
Yes, 100%. I’ve been saying this for a long time… the data is everything.
-
AI Writing Excellence and Software Operating Agent Development
By
–
Two main things: 1. We are the best AI for writing and writing-related tasks. Our models write naturally, have access to up-to-date info, can research + cite sources. + lots more. 2. We’ve been developing an AI agent that can operate software to get tasks done for you.
-
Dataset Quality: The Foundation of AI Model Performance
By
–
The dataset is everything. Great read: https://
nonint.com/2023/06/10/the
-it-in-ai-models-is-the-dataset/
… -
Solving Long-Context LLaMA 3 by End of Day
By
–
By the end of today, I think we will have solved long-context LLaMA 3.
-
Fast Generation in Large Models with Sparse Parameter Activation
By
–
Not sure how it will play out, but it means that generations will be very fast for such a big model, as only a small portion of params will be active at a time.
-
Snowflake’s New Development: Enterprise AI Adoption Challenges
By
–
I’m not sure most companies have a use for this (we certainly won’t use it at HyperWrite, due to the sheer size / performance ratio). But still a really cool development for the Snowflake team!