The number of logprobs returned per token isn’t the issue here, it’s the unavoidable difficulty of getting logprobs for different completion sequences. In a classification context you need to assign class labels to individual tokens during training.

→ View original post on X — @goodside