This is why I want to see an example – just one – of a successful fine-tune that added specific extra knowledge to a model Until I've seen that I can't say for certain that it can be done
@simonw
-
Model Weight Editing: Integrating Knowledge Beyond RAG
By
–
Editing weights is super interesting too, basically anything where the model weights themselves have been updated to introduce new knowledge in place of using RAG for that
-
Fine-tuning for Knowledge: Challenging Current Mental Models
By
–
Yeah they did – I'm looking for anything that can challenge my current mental model that fine-tuning for extra knowledge isn't worthwhile
-
Tiktoken Tokenization: Why Not Handle Unknown Characters Gracefully?
By
–
Something I don't understand here is that surely it would make sense to maintain the ability to tokenize as a sequence of integer characters representing the pieces that make up that string, rather than throwing an error? Any idea why tiktoken doesn't do that?
-
Fine-tuned Models for Additional Knowledge: Current Examples
By
–
Yeah, that's pretty much what I figured – at this point I'm wondering if there are ANY examples of fine-tuned models to serve additional knowledge out there
-
ttok 0.3 releases new allow-special option for tokenization
By
–
New release of ttok (version 0.3) adds a "–allow-special" option that fixes that: https://
github.com/simonw/ttok/re
leases/tag/0.3
… -
OpenAI tiktoken library rejects special tokens by default
By
–
TIL the OpenAI tikoken library rejects text that includes special tokens by default, which affects my own "ttok" CLI tool – it raises an error if you call it like this: ttok '' –encode
-
Escaping Implementation Guidance Security Best Practices
By
–
That's what I'm looking for here – guidance on how that kind of escaping should be implemented, ideally backed up by demonstrations of what goes wrong if you don't and how the proposed escaping mechanism prevents those issues
-
Fine-tuning models for knowledge addition and RAG applications
By
–
That's not quite what I'm looking for – I believe that fine-tuning can work for things like adjusting the tone of the model's responses, the thing I'm seeking evidence for is fine-tuning in order to add additional knowledge to a model such that it can be used for RAG-style Q&A
-
Fine-tuning models for knowledge acquisition beyond RAG
By
–
Phind is running RAG – you can tell from the sources it displays in the side-bar They may have fine-tuned a model for being more effective at RAG, but that's not what I'm looking for here – I want an example of a model that was fine-tuned in order to bake additional knowledge