This abstraction also opens up the door to returning more information about each generation For example, the logprobs associated with a generation (something I know @alexgraveley would love)
New abstraction enables returning generation metadata like logprobs
By
–
Leave a Reply