There’s conditioning from the dialog syntax that it’s being naively given in the same format that the instruct-tuned version receives. It’s seen these in pre-training, but the association isn’t strong enough apparently to make it act like a chatbot even most of the time.
RESEARCH
-
Example contrasting base LLMs and RLHF-tuned models
By
–
I tried to write a prompt to show how base LLMs differ from the RLHF-tuned ones everyone knows, and I think this gives a bit of the flavor. A message from Llama 3.1 405B (base), on whether it’s useful to talk to base LLMs:
-
FocusLLM: Scaling LLM Context Through Parallel Decoding
By
–
FocusLLM Scaling LLM's Context by Parallel Decoding discuss: https://
huggingface.co/papers/2408.11
745
… Empowering LLMs with the ability to utilize useful information from a long context is crucial for many downstream applications. However, achieving long context lengths with the conventional -
Holistic Evaluation Framework for Video Foundation Models
By
–
TWLV-I Analysis and Insights from Holistic Evaluation on Foundation Models discuss: https://
huggingface.co/papers/2408.11
318
… In this work, we discuss evaluating video foundation models in a fair and robust manner. Unlike language or image foundation models, many video foundation models -
TrackGo: Flexible Method for Controllable Video Generation
By
–
TrackGo
— AK (@_akhaliq) 22 août 2024
A Flexible and Efficient Method for Controllable Video Generation
discuss: https://t.co/I8plJvUuUC
Recent years have seen substantial progress in diffusion-based controllable video generation. However, achieving precise control in complex scenarios, including… pic.twitter.com/oOWby6e6pBTrackGo A Flexible and Efficient Method for Controllable Generation discuss: https://
huggingface.co/papers/2408.11
475
… Recent years have seen substantial progress in diffusion-based controllable video generation. However, achieving precise control in complex scenarios, including -
Discussion on RLHF vs post-training model comparisons
By
–
You’re not comparing RLHF to no RLHF here, you’re comparing different generations post-RLHF mystery models, likely of different sizes. If you don’t do post-training at all, naive attempts to talk to the model go like this:
-
Einstein Derives OODA Loop from Fundamental Universe Equations
By
–
Here Albert Einstein is deriving the OODA Loop from the fundamental equations that define our universe. #ooda #oodaloop
-
Google Researchers Gebru and Mitchell Dismissed: Correction
By
–
Petit point de correction: je n'ai jamais travaillé chez Google. C'est mes co-autrices (Gebru et Mitchell) qui sont licenciées.
-
Neuralink implants scaling to millions within decade
By
–
Update about the second Neuralink device in a human. If all goes well, there will be hundreds of people with Neuralinks within a few years, maybe tens of thousands within 5 years, millions within 10 years, …
-
FOXY Method: Self-Improving Agents Design Pattern
By
–
I'm starting to see some more projects and research around self-improving agents, so thought I'd reshare this design pattern I introduced last year. The "FOXY method" tracks output, generates a reflection, and feeds the most relevant ones when running future objectives, to help