Yeah — simply multiplied rope * 2, trained on long data with a MSL of 16K.
AI
-
LLaMA 3 70B enables instant code refactoring on Groq
By
–
Using LLaMA 3 70B on @GroqInc to instantly refactor and document code.
— Matt Shumer (@mattshumer_) 22 avril 2024
The implications for software engineering are wild.
Gone are the days of waiting on an LLM for suggestions or code changes.
Now, it's an instant feedback loop.
Demo link in the comments: pic.twitter.com/joLJy4oJ3TUsing LLaMA 3 70B on @GroqInc to instantly refactor and document code. The implications for software engineering are wild. Gone are the days of waiting on an LLM for suggestions or code changes. Now, it's an instant feedback loop. Demo link in the comments:
-
LLM Plugin Directory Now Features 30 Available Plugins
By
–
The LLM plugin directory lists 30 plugins now! https://
llm.datasette.io/en/stable/plug
ins/directory.html
… If you want to try writing your own there's a comprehensive tutorial included in the docs: https://
llm.datasette.io/en/stable/plug
ins/tutorial-model-plugin.html
… -
LLaMA 3-based model deployment nearing completion
By
–
Very soon. This is one of the last things I have to get working well before we can push a LLaMA 3-based model live.
-
Extending Context Length for Base Model Development
By
–
Planning to do this for the base model, at least at first. It's more of a learning thing for me, to help me perfect the context length extension approach so I can do it for our internal HyperWrite stuff… so I don't want to do too many variations.
-

Llama 3 Announcement Highlights Data Quality’s Impact on AI Performance
By
–
Llama 3 announcement says the quality of both SFT and preference data have an “outsize influence” on performance. The first excerpt above alludes to an explanation for why preferences would help with reasoning. (This screenshot is the text just before the one above:)
-
LLaMA 3 8B Extended to 16K Token Context Window
By
–
Training a LLaMA 3 8B that supports 16K tokens, twice the current context window. If successful, will open-source it.
-

Nvidia plummets: TL20 index records worst day ever
By
–
The TL Podcast for April 21st: Nvidia takes a tumble Nvidia fell on Friday by the most in the two and a half years of the TL20, giving the group its worst day ever. In the absence of any particular news, this event was a healthy right-sizing. Nvidia still has no real competition
-
EU Warns TikTok Lite Over Addiction and Harmful Content Risks
By
–
The EU warned it may suspend TikTok Lite, a new app from the company that offers users “rewards” for engaging with content, if it doesn’t sufficiently address potential risks around addiction and harmful content.