So, how can you do this? By adjusting the model parameters. There are 3 parameters you can adjust:
1. Temperature
2. Top-k
3. Top-p But before we see how to use them, let’s first understand how the model selects the next token to generate.
LLMS
-
Adjusting Model Parameters: Temperature, Top-k, Top-p
By
–
-
FHE Limitations for Large Model Inference Privacy
By
–
But FHE is not so practical for these models. And it may be infeasible to use some of these large models on a user's device. So even with private fine-tuning, then privacy at inference time still remains. 11/n
-
ML Privacy Risks: Models Can Leak Secrets from Training Data
By
–
There's more such examples in large pre-training datasets. Since ML models pretrained on them without privacy considerations can be coerced to spit secrets out verbatim (see e.g., https://
arxiv.org/abs/2012.07805 by Carlini et al), this dilutes the meaning of "privacy" for such models 7/n -
Private Fine-Tuning of LLMs Shows Modest Utility Loss
By
–
Story is similar for language models. In some prior works at #ICLR2022 (by Yu et al https://
arxiv.org/abs/2110.06500 and @lxuechen et al https://
arxiv.org/abs/2110.05679), it was shown that privately fine-tuning (publicly) pretrained LLMs suffers only a modest utility loss. 4/n -
Seeking source for GPT-3 family tree graphic
By
–
great graphic – is there a canonical source for the gpt3 family tree or did you just come up with this? (want to make sure i cite properly when i definitely use this in future)
-
RLHF automates human evaluation and generates tuning data
By
–
Later, reinforcement learning was used to automate the task of human evaluation as well. This yielded a endless fountain of tuning data that could be produced entirely by machine — a process known as RLHF.
-
Tuning models on filtered self-generations for smarter AI
By
–
By tuning the model on its own generations, filtered to those deemed perfect by human evaluators, greater volumes of data could be used, yielding a more intelligent and obedient model.
-
Human demonstrations limited early instruction tuning models
By
–
Early attempts at instruction tuning relied entirely on demonstrations from humans. This made the model easier to prompt, but the approach was limited by the inherent difficulty of manufacturing new humans.
-
Model Evaluations Counting Details Fine Print Clarification
By
–
Good point – the fine print states we're actually counting model evals
-
Instruction tuning merges human will into AI controlled by awkward kid
By
–
Instruction tuning / RLHF is technically a Human Instrumentality Project, merging the preferences of countless humans to form an oversized, living amalgam of our will. We then hand control of it to a random, socially awkward kid and hope for the best.