60 tokens a second, context window is supposedly up to 262,144 but that will use a ton more RAM, probably more than my Mac can handle (I've not tried though)
Context Window Performance and RAM Requirements for AI Models
By
–
Global AI News Aggregator
By
–
60 tokens a second, context window is supposedly up to 262,144 but that will use a ton more RAM, probably more than my Mac can handle (I've not tried though)
Leave a Reply