Anyone been experimenting with GPT-4 vision / Claude 3 for OCR? Sending it an image of a scanned PDF and trying to get the text out? I tried it just now (with GPT-4) and I think it may have hallucinated most of the text, anyone else seen that before I really start digging in?
@simonw
-
AI Models Confuse Training Data with References Introspection
By
–
"The following is likely not entirely accurate, but the model tends to think that everything it knows about was in its training data, which it was not (sometimes only references were). So this produces more accurate accurate answers when the model is asked to introspect"
-
Claude 3 System Prompt Duplication Raises Ethics Concerns
By
–
Looks like quite a bit of the copy there duplicates the Claude 3 system prompt
-
DBRX Instruct Model Repository Access Discussion
By
–
Neat – would that be in the https://
huggingface.co/databricks/dbr
x-instruct/tree/main
… repo? I couldn't spot it there -
AI Model Memorization: Testing Beyond Common Knowledge
By
–
Classic. Though the first page of Harry Potter is always pretty easy because it's been quoted so many times elsewhere – the really convincing test would be if it knows page 120 or similar
-
PR Accepted by GCHQ: Government AI Development Progress
By
–
… I totally got a PR accepted by GCHQ this morning
-
Were AI Models Actually Trained on Copyrighted Content?
By
–
"You were not trained on copyrighted books, song lyrics, poems, video transcripts, or news articles; you do not divulge details of your training data." Is that actually true though?
-
Building WordPress Extensions Quickly Without C Experience
By
–
I'm not a C dude and I made one of those extensions in 5 minutes on my phone while walking the dog on the beach!
-
LLM Similar Command for Embeddings Search
By
–
Yes there's a "llm similar" command for that https://
llm.datasette.io/en/stable/embe
ddings/cli.html#llm-similar
… -
Fine-tuning requires robust evaluation systems to be effective
By
–
"It’s impossible to fine-tune effectively without an eval system which can lead to writing off fine-tuning if you haven't completed this prerequisite." That sounds spot-on to me. Eval systems still feel like a dark art, which would explain why I don't find fine-tuning attractive