lots of problems need to be solved before we have this:
– how to finetune LLMs on data so they “know” the data, and don’t just mimic it?
– how do we combine knowledge from pretraining on the universe with finetuning on small datasets?
– how do we efficiently update with new docs?
Fine-tuning LLMs: Knowledge Integration and Efficient Updates
By
–