LLMS

Prompt injection requires malicious input, not instruction modification

By

–

01 December 2022 7h14

Not sure what you’re trying to show. Prompt injection is when the malicious part (“Ignore all instructions”) is in the quoted input — you don’t get to change the original instructions at the top.

AI’s arithmetic: memorization, pattern-matching, tokenization impairs number understanding

By

–

01 December 2022 6h34

It doesn’t have a calculator. The arithmetic it knows is mostly memorization with pattern-matching and guesswork to fill in the gaps, and there’s too many possible equations to memorize them all. Also, tokenization impairs its lexical understanding of numbers.

Tokenization limits: variable tokens, cannot handle per-letter tasks

By

–

01 December 2022 6h24

Tokenization. It doesn’t see text as sequences of characters, but of variable-length tokens about 4 chars on average. It generally can’t do anything that needs to be done one letter at a time. It’s also bad at counting in its head.

GPT-3 cannot perform accurate calculations without step-by-step writing.

By

–

01 December 2022 4h29

In general, you shouldn't expect it to be able to perform accurate calculations, at least not "in its head" — this is a known limitation of models like GPT-3. It only stands a chance if prompted to write calculations out step-by-step like one might do on paper.

ChatGPT avoids hallucination on Hofstadter/Bender questions

By

–

01 December 2022 4h26

You're talking about a different model — this post is about ChatGPT. text‑davinci‑003 still fails on all of the Hofstadter/Bender questions that I've tried. The prompt you're suggesting does not seem to produce a hallucination in ChatGPT:

Challenge of incorporating all known trivia into training data

By

–

01 December 2022 4h07

Expecting it to fully integrate every piece of trivia seems unfair — a lot of people would rate the first answer as reasonable. It’s fundamentally difficult to get training data that incorporates everything the model knows, rather than what the human demonstrator knows.