I haven't run real benchmarks on it (if anyone wants to, that'd be awesome!), but from my homemade haystack tests, the context length extension worked well.
AI
-

LLaMA 3 Context Window Doubled to 16K Tokens
By
–
I've doubled LLaMA 3's context window to 16K tokens. Fully open-source. Link in thread:
-
LLMs as Arrays: Meditation on Entropy and Truth
By
–
As I emerged from meditation it dawned on me that LLMs are just one array of floats and a while loop over some super simple arithmetic on its elements. It is entropy that is the root of suffering. It's by deleting the superfluous that we uncover truth. And thus I was enlightened.
-

Goal-Play AI Simulations for Interactive Learning
By
–
Here is a goal-play prompt you can try. In a goal play, the AI sets up a scenario in which the student knows something the character in the simulation does not and guides the character using what they know, GPT: https://
chat.openai.com/g/g-Rg4AbuKui-
goal-setting-goal-play
…
Gemini Advance: https://
g.co/gemini/share/0
960efc4192b
… -
Evaluation Loss in Model Training: Understanding Its Role
By
–
eval loss. i don't always use it, as it's not a great measure, but it's good for things like this
-
ICLR Features Hugging Face Demo Platform for AI Models
By
–
ICLR blog: https://
blog.iclr.cc/2024/04/22/hug
ging-face-demo-site/
… -

Hugging Face Demo Site for ICLR Conference Paper Submissions
By
–
Hugging Face is hosting a demo site for @iclr_conf authors to find and claim their papers and discuss those papers on dedicated pages The page includes all accepted papers, along with linked datasets, models, and demos
-
Common Research Pitfall: Confounding Variables in Experiments
By
–
common dark thought pattern in research
> run baseline experiment
> change thing A
> also change thing B
> run new experiment
> collect results
> "wow, thing A works!" -
Lean Co-pilot Reduces Effort for Theorem Proving Tasks
By
–
Our recent Lean Co-pilot significantly reduces human effort needed for theorem proving in Lean
-
Lean Copilot Automates Over 80% of Mathematical Proof Steps
By
–
We have now posted an expanded version of our Lean Co-pilot paper and updated the code base. https://
arxiv.org/abs/2404.12534 https://
github.com/lean-dojo/Lean
Copilot
…
The credit goes to @p_song1 for driving this effort.
Our latest experiments show that our co-pilot can automate >80% of the proof steps