Not that I've seen yet, but you probably don't want to do that – as a general rule fine-tuning on a small number of documents isn't an effective way to "add knowledge" to a model – you should look at implementing RAG instead Some notes on that here:
LLMS
-
Reactor Migration to Claude Opus 3: Triple Cost for Triple Quality
By
–
When we moved Reactor to Claude Opus 3 our costs tripled but so did the quality of the outputs
-
LLaMA 3 Context Window Doubled to 16K Tokens Open-Source
By
–
I've doubled LLaMA 3's context window to 16K tokens. Fully open-source. Link in thread:
-
LLaMA 3 Context Window Extended to 16K Tokens Open-Source
By
–
I've doubled LLaMA 3's context window to 16K tokens. Fully open-source. Link in thread:
-
LLaMA 3 Context Window Doubled to 16K Tokens
By
–
I've doubled LLaMA 3's context window to 16K tokens. Fully open-source. Link in thread:
-
Context Length Extension Testing and Performance Benchmarks
By
–
I haven't run real benchmarks on it (if anyone wants to, that'd be awesome!), but from my homemade haystack tests, the context length extension worked well.
-
LLaMA 3 Context Window Doubled to 16K Tokens
By
–
I've doubled LLaMA 3's context window to 16K tokens. Fully open-source. Link in thread:
-
LLMs as Arrays: Meditation on Entropy and Truth
By
–
As I emerged from meditation it dawned on me that LLMs are just one array of floats and a while loop over some super simple arithmetic on its elements. It is entropy that is the root of suffering. It's by deleting the superfluous that we uncover truth. And thus I was enlightened.
-
Evaluation Loss in Model Training: Understanding Its Role
By
–
eval loss. i don't always use it, as it's not a great measure, but it's good for things like this
-
Lean Co-pilot Reduces Effort for Theorem Proving Tasks
By
–
Our recent Lean Co-pilot significantly reduces human effort needed for theorem proving in Lean