for those who haven't heard about this:
openAI only gives you the top 5 logprobs per response (max). we (mostly @justintchiu
) came up with an algorithm to get logits from openAI. essentially for a different token (not top-5), u can binary search logit bias to see the minimal

→ View original post on X — @jxmnop