Dude…. big fan. Your character-level GPT models were how I initially got into training models
@mattshumer_
-
Cursor AI Team Likely Developing Similar Features
By
–
I'm sure the @cursor_ai team is thinking about something similar
-
RoPE Scaling Enables 16K Token Context Window Models
By
–
Yeah — simply multiplied rope * 2, trained on long data with a MSL of 16K.
-
LLaMA 3 70B enables instant code refactoring on Groq
By
–
Using LLaMA 3 70B on @GroqInc to instantly refactor and document code.
— Matt Shumer (@mattshumer_) 22 avril 2024
The implications for software engineering are wild.
Gone are the days of waiting on an LLM for suggestions or code changes.
Now, it's an instant feedback loop.
Demo link in the comments: pic.twitter.com/joLJy4oJ3TUsing LLaMA 3 70B on @GroqInc to instantly refactor and document code. The implications for software engineering are wild. Gone are the days of waiting on an LLM for suggestions or code changes. Now, it's an instant feedback loop. Demo link in the comments:
-
LLaMA 3-based model deployment nearing completion
By
–
Very soon. This is one of the last things I have to get working well before we can push a LLaMA 3-based model live.
-
Extending Context Length for Base Model Development
By
–
Planning to do this for the base model, at least at first. It's more of a learning thing for me, to help me perfect the context length extension approach so I can do it for our internal HyperWrite stuff… so I don't want to do too many variations.
-
LLaMA 3 8B Extended to 16K Token Context Window
By
–
Training a LLaMA 3 8B that supports 16K tokens, twice the current context window. If successful, will open-source it.
-
Data Congruency Critical for Fine-Tuning LLMs in Products
By
–
When fine-tuning LLMs for products, data congruency is so important.
-
Rope Scaling: Advanced Approaches for LLM Optimization
By
–
Many approaches — look into Rope scaling as a start.