GENERATIVE AI

Tokenization limits: variable tokens, cannot handle per-letter tasks

By

–

01 December 2022 6h24

Tokenization. It doesn’t see text as sequences of characters, but of variable-length tokens about 4 chars on average. It generally can’t do anything that needs to be done one letter at a time. It’s also bad at counting in its head.

GPT-3 cannot perform accurate calculations without step-by-step writing.

By

–

01 December 2022 4h29

In general, you shouldn't expect it to be able to perform accurate calculations, at least not "in its head" — this is a known limitation of models like GPT-3. It only stands a chance if prompted to write calculations out step-by-step like one might do on paper.

ChatGPT avoids hallucination on Hofstadter/Bender questions

By

–

01 December 2022 4h26

You're talking about a different model — this post is about ChatGPT. text‑davinci‑003 still fails on all of the Hofstadter/Bender questions that I've tried. The prompt you're suggesting does not seem to produce a hallucination in ChatGPT:

Challenge of incorporating all known trivia into training data

By

–

01 December 2022 4h07

Expecting it to fully integrate every piece of trivia seems unfair — a lot of people would rate the first answer as reasonable. It’s fundamentally difficult to get training data that incorporates everything the model knows, rather than what the human demonstrator knows.

GPT-3 improvement shows not just pattern-matching, still hallucinates details

By

–

01 December 2022 3h44

The point isn’t that it’s perfect, just that it’s not narrowly pattern-matching on this specific list of questions — there’s clear improvement vs. GPT-3 across many questions that contain false assumptions. It does still hallucinate details in other ways.

OpenAI ChatGPT explains bubble sort complexity in 1940s gangster style

By

–

01 December 2022 2h39

OpenAI's new ChatGPT explains the worst-case time complexity of the bubble sort algorithm, with Python code examples, in the style of a fast-talkin' wise guy from a 1940's gangster movie:

→ View original post on X — @nandodf

Generative Models Impress with Accessibility and Creative Potential

By

@nandodf

–

01 December 2022 1h29

This application of generative models is one of the most impressive AI demos I’ve ever seen. I’m amazed at the results but also at how unexpectedly easy, creative and accessible these tools have become. We do live in special times.