I've doubled LLaMA 3's context window to 16K tokens. Fully open-source. Link in thread:
CODE
-
LLaMA 3 Context Window Doubled to 16K Tokens
By
–
I've doubled LLaMA 3's context window to 16K tokens. Fully open-source. Link in thread:
-
Evaluation Loss in Model Training: Understanding Its Role
By
–
eval loss. i don't always use it, as it's not a great measure, but it's good for things like this
-
Lean Co-pilot Reduces Effort for Theorem Proving Tasks
By
–
Our recent Lean Co-pilot significantly reduces human effort needed for theorem proving in Lean
-
Lean Copilot Automates Over 80% of Mathematical Proof Steps
By
–
We have now posted an expanded version of our Lean Co-pilot paper and updated the code base. https://
arxiv.org/abs/2404.12534 https://
github.com/lean-dojo/Lean
Copilot
…
The credit goes to @p_song1 for driving this effort.
Our latest experiments show that our co-pilot can automate >80% of the proof steps -
Backends Simplified: LLM and Prompt Engineering Revolution
By
–
Well it’s true that the backends would be a simple prompt + a LLM nowadays …
-
Ollama LLM Plugin Integration for WordPress
By
–
Yes, ollama is a great option too – I only wrote about the options I'd got running myself, ran out of time! There's an LLM plugin for ollama here:
-
Claude generates buggy 3D spaceship simulator game code
By
–
I asked Claude to make me a 3D spaceship simulator game and it produced this code, which I pasted into a p5.js editor online as instructed.
— Ethan Mollick (@emollick) 22 avril 2024
It is obviously full of strange bugs that I could work with Claude to solve. But controls work and I appreciate the sheer weirdness. pic.twitter.com/6wRFsmU0jSI asked Claude to make me a 3D spaceship simulator game and it produced this code, which I pasted into a p5.js editor online as instructed. It is obviously full of strange bugs that I could work with Claude to solve. But controls work and I appreciate the sheer weirdness.
-
Cursor AI Team Likely Developing Similar Features
By
–
I'm sure the @cursor_ai team is thinking about something similar
-
RoPE Scaling Enables 16K Token Context Window Models
By
–
Yeah — simply multiplied rope * 2, trained on long data with a MSL of 16K.