I believe that’s just a product decision by the API providers. No reason that can’t be extended. At HyperWrite, we often offer 5000 to 10,000 token outputs to users.
@mattshumer_
-
Master Prompt Engineering Through Practice and Challenges
By
–
It’s difficult to get really good at prompt engineering by reading. Just write prompts, write more prompts, and then write even more prompts. Come up with challenges for yourself and try to make it work. You’ll get better faster than you can imagine.
-
Extending L3 70B context length with Unsloth on single A100
By
–
I almost always use (and LOVE) Axolotl… but wanted to try Unsloth to try to extend L3 70B's ctxlen w/ just one A100.
-
LLaMA 3-Instruct Dataset Format Structure Guide
By
–
Specifically, I have a dataset in the LLaMA 3-Instruct format:
“`
system {{ system_prompt }}user {{ user_message_1 }}assistant {{ -
LLaMA 3 Instruct Format Dataset Structure
By
–
Specifically, I have a dataset in the LLaMA 3-Instruct format:
“`
system {{ system_prompt }}user {{ user_message_1 }}assistant {{ -
AI Systems Can Learn New Knowledge With Proper Implementation
By
–
False. It can learn new knowledge if done right.
-
LLaMA 3 Reaches 160K+ Context Window, Continuing Rapid Expansion
By
–
LLaMA 3, now with 160K+ context. Every day, we are pushing it higher.
-
SFTTrainer Response Template Training Logic Issue
By
–
Is it just me, or does the @huggingface SFTTrainer's approach to training only on completions make no sense? What if your response template shows up multiple times in your example? Am I misunderstanding this?
-
Fine-tuning Models with Detailed Instructions for Faster Learning
By
–
Basically, in your fine-tuning data, include detailed instructions on how to behave — don't make the model guess at it. This way, it learns faster.
-
LLaMA 3 Extended Context Window Reaches 128K Tokens
By
–
It's been a week since LLaMA 3 dropped. In that time, we've:
– extended context from 8K -> 128K
– trained multiple ridiculously performant fine-tunes
– got inference working at 800+ tokens/second If Meta keeps releasing OSS models, closed providers won't be able to compete.