Phi-3 model weights coming very soon
LLMS
-
GPT-3.5 prompt injection defense and fine-tuning advantages
By
–
I quite like that they're using GPT3.5 here – for prompt injection a defense that works with the less expensive, faster models would almost certainly work with the more expensive ones too I imagine GPT-3.5 is better for fine tuning at the moment because there's more experience
-
51 Posts on Prompt Injection: Complete Collection
By
–
Here are all 51 of my posts on prompt injection so far – love having tags on my blog!
-

OpenAI releases detailed prompt injection evaluation paper
By
–
New paper from @OpenAI on prompt injection – it's the most detailed evaluation of the problem I've seen from them so far, and has some very interesting details Posted some of my notes on the paper on my log here: https://
simonwillison.net/2024/Apr/23/th
e-instruction-hierarchy/
… -
Experimenting with thinking in other languages for AI research
By
–
Thinking in other languages is a fascinating idea. I want to experiment with that. Very clever.
-
Larger Models Required for Quality Content Creation
By
–
For content creation specifically we need that magic that only comes from bigger models
-
Fine-tuning vs RAG: Choosing the Right Approach
By
–
If you get that working I'd love to hear about it! I've been mostly put off fine-tuning for this kind of thing because RAG feels easier to get results out of
-

Phi-3 Announced: GPT-3.5 Rival Runs on Smartphones
By
–
Phi-3 was just announced! Benchmarks rival GPT-3.5, but the model can be run on a phone. BUT — as with all (tiny) models, benchmarks tell us less than vibes.
-
Training costs for 8B parameter models relatively affordable
By
–
Couple hundred. Pretty cheap b/c only 16K on 8B
-
70B Model Context Window Limitations and Hardware Requirements
By
–
Ooof. I don't know if 128K is possible on the 70B, unless someone can get me a rig with like 16 or more A/H100s. 32K is likely doable though.