At higher temperatures (>1.25) GPT-4's writing will often dissolve into nonsense, and that risk increases the longer it writes. You can also use the top-p parameter in tandem (try 0.9 to 0.99) to get more creativity without it going completely off the rails.
Tuning GPT-4 Output: Temperature and Top-p Parameters for Creativity
By
–