That's great, thanks! I'll continue to hold out on my dream for daily spending limits – I may have to implement that myself, though it would be easier if you fix the bug where streaming responses don't get valid "usage" JSON blocks
GENERATIVE AI
-
Fine-tuned Models Still Vulnerable to Adversarial Attacks
By
–
TLDR version: the fine-tuned model they describe improves things, but I still don't think "improves" is good enough If you are facing an adversarial attacker then reducing the chance that they might find an exploit just means they’ll try harder until they find one that works
-
OpenAI releases detailed prompt injection evaluation paper
By
–
New paper from @OpenAI on prompt injection – it's the most detailed evaluation of the problem I've seen from them so far, and has some very interesting details Posted some of my notes on the paper on my log here: https://
simonwillison.net/2024/Apr/23/th
e-instruction-hierarchy/
… -
ChatGPT struggles with complex visual scene generation
By
–
ChatGPT struggles a bit:
"Visual Elements: Towering trees, a dimly lit path splitting into two, fading dusk light, shadows moving slightly with the wind." Its code drew this (to be fair, I think it "wanted" the tress to be vertical cones) -
Claude 3 Fantasy Game with p5js 3D Animation Scenes
By
–
"Claude 3, we are going to play a fantasy game. you are the dungeon master. you will give me a scenario and illustrate each scene with a p5js 3D animation I can just paste right into an editor…."
— Ethan Mollick (@emollick) 23 avril 2024
Claude: "You soon find yourself in a circular grove, surrounded by ancient trees" pic.twitter.com/MMkUNGu6vy"Claude 3, we are going to play a fantasy game. you are the dungeon master. you will give me a scenario and illustrate each scene with a p5js 3D animation I can just paste right into an editor…." Claude: "You soon find yourself in a circular grove, surrounded by ancient trees"
-
Larger Models Required for Quality Content Creation
By
–
For content creation specifically we need that magic that only comes from bigger models
-
Fine-tuning vs RAG: Choosing the Right Approach
By
–
If you get that working I'd love to hear about it! I've been mostly put off fine-tuning for this kind of thing because RAG feels easier to get results out of
-
Phi-3 Announced: GPT-3.5 Rival Runs on Smartphones
By
–
Phi-3 was just announced! Benchmarks rival GPT-3.5, but the model can be run on a phone. BUT — as with all (tiny) models, benchmarks tell us less than vibes.
-
Training costs for 8B parameter models relatively affordable
By
–
Couple hundred. Pretty cheap b/c only 16K on 8B
-
Fine-tuning vs RAG: Best Practices for Model Knowledge
By
–
Not that I've seen yet, but you probably don't want to do that – as a general rule fine-tuning on a small number of documents isn't an effective way to "add knowledge" to a model – you should look at implementing RAG instead Some notes on that here: