In SFT we often train on other agents data (distillation) so you’re right. However we also try to climb by self-improvement. This is where it becomes important for the agent not to train on its actions. Any bias on the agent beliefs (weights of the neural net) will be amplified.
Avoiding self-training bias in agent self-improvement
By
–