Specifically, I have a dataset in the LLaMA 3-Instruct format:
“`
system {{ system_prompt }}user {{ user_message_1 }}assistant {{
LLMS
-
LLaMA 3 Instruct Format Dataset Structure
By
–
-
AI Systems Can Learn New Knowledge With Proper Implementation
By
–
False. It can learn new knowledge if done right.
-
LLaMA 3 Reaches 160K+ Context Window, Continuing Rapid Expansion
By
–
LLaMA 3, now with 160K+ context. Every day, we are pushing it higher.
-
SFTTrainer Response Template Training Logic Issue
By
–
Is it just me, or does the @huggingface SFTTrainer's approach to training only on completions make no sense? What if your response template shows up multiple times in your example? Am I misunderstanding this?
-
Large Language Models as Agents: Action Prediction Framework
By
–
One intuitive way to think about how AI models (Large Language Models) can are used as an "agent" is to consider using the model to predict what actions to take, given a particular state of the world. This approach some times go by the name "Large Action Model". For example, you
-
Advanced Vector Retrieval with Graph-Based Metadata Using LangChain Neo4j
By
–
Optimizing vector retrieval with advanced graph-based metadata techniques using LangChain and Neo4j Text embeddings and vector similarity search help us find documents by understanding their meanings and how similar they are to each other. However, text embeddings aren’t
-
Volume 239: Foundation Models Failure Modes Proceedings Published on PMLR
By
–
Volume 239 Proceedings on "I Can't Believe It's Not Better: Failure Modes in the Age of Foundation Models" https://
proceedings.mlr.press/v239/ Is now available on PMLR. -
Fine-tuning Models with Detailed Instructions for Faster Learning
By
–
Basically, in your fine-tuning data, include detailed instructions on how to behave — don't make the model guess at it. This way, it learns faster.
-
LLaMA 3 Extended Context Window Reaches 128K Tokens
By
–
It's been a week since LLaMA 3 dropped. In that time, we've:
– extended context from 8K -> 128K
– trained multiple ridiculously performant fine-tunes
– got inference working at 800+ tokens/second If Meta keeps releasing OSS models, closed providers won't be able to compete. -
System Prompts Quality Boosts Model Fine-Tuning Efficiency
By
–
High alpha trick for fine-tuning: Make your system prompts in your dataset really great. It'll help the model learn to do your task much faster, with less data. If you have lots of data, you can ignore this, but at small dataset sizes, this changes everything.