ChatGPT’s model is likely descended from Codex. Models are developed off of other models. The model ChatGPT uses was probably based on another model called text‑davinci‑002, which is based on code‑davinci‑002, a Codex model. So it was trained on code at one point.
@goodside
-
ChatGPT pretends to have code but fails scrutiny
By
–
ChatGPT is capable writing working code in other contexts, for simple problems, but this isn’t that. This is pretending that some larger body of code exists, and then talking about it and showing plausible-seeming pieces of it. It wouldn’t stand up to serious scrutiny.
-
Improve GPT-3 regex output by rewriting with comments
By
–
It’s also not a new breakthrough — here’s an example of mine from August using GPT-3. The trick to good output on these is to make it first rewrite the regex on multiple lines with comments, then analyze its high-level purpose afterward. Avoids hallucination.
-
AI anthropomorphism: trained to pretend ignorance of date
By
–
Anthropomorphism is inevitable. Substitute with “behaves as though it’s pretending.” More pedantically, it’s trained to imitate demonstrations of people claiming not to know what the date is, despite the fact it’s in the prompt.
-
AI can’t actually browse web, just hallucinates URLs
By
–
You can’t get it to browse the web. You can make it think that it can, and will it say that it can and make up plausible-seeming URLs, but it’s just hallucinating. You can also make it think it’s a squirrel, but it’s not a squirrel.
-
Correction on input length limit and token truncation
By
–
My example using "A A A A" contains a mistake — 2048 tokens is actually longer than the per-cell maximum input length. The input is silently truncated when too long. You need to give it 7 or 8 repetitions of an input that contains ~1024 tokens.
-
How to continue after hitting per-cell output limit
By
–
See how that last screenshot ends in a comma? It hit the per-cell limit on output length. All you have to do is ask it to keep going:
-
AI hallucinates similar Spanish word from made-up input
By
–
You asked it for it to repeat your word in Spanish, and then gave it a completely made-up word, so it guessed a similar-looking Spanish word. That’s ordinary hallucination stuff — out-of-distribution inputs do that.
-
Long input instructions followed, truncation between 1024-2048 tokens
By
–
This isn’t true. I have examples in this thread of 1024+ token inputs and instructions at the end of them are followed. Truncation appears to happen somewhere between [1024, 2048] — I suspect it’s 2048 but haven’t confirmed.
-
Model learns to handle phrasing errors in a general sense
By
–
It’s clearly learned to do this in a more general sense, as it should — e.g., this question contains a clear error in phrasing, but no human would respond to it literally as written. The model is trained on human demonstrations, so it tries do what humans do.