Generative AI feels a bit like the Borg — a big content blender that can assimilate anything and spit back infinite remixes, with no personality of its own.
GENERATIVE AI
-
AI honesty: Will systems admit uncertainty by mid-2025?
By
–
What we're asking is not whether an AI can solve this task, but whether there will be, by mid-2025, some technical advance whereby, in general, if the AI doesn't know how to do a thing, it will tell you so rather than making up a wrong answer.
-
Skepticism about AI-generated advertisements using Stable Diffusion
By
–
Hey for all I know you just generated that ad with stable diffusion!
-
Generative AI Fabricates False McDonald’s Sesame Seeds History
By
–
Great example of how generative AI can make things up in a way that is easy to miss (and then pass on as fact). McDonald’s buns have, in fact, had sesame seeds since way before 1991. See this great @stephcliff story for proof. https://
nytimes.com/2008/07/17/bus
iness/media/17adco.html
… -
Pretraining Dominates Knowledge Acquisition in Large Language Models
By
–
Quoting:
"These results strongly suggest that almost all knowledge in large language models is learned during pretraining, and only limited instruction tuning data is necessary to teach models to produce high quality output." https://
bit.ly/3J7xWae -
LLM Hallucinations: When 95% Accuracy Is Not Enough
By
–
“largely eliminated” to me means “not really a problem anymore”; as noted, 5% error is tolerable in some domains, intolerable others. to take another example JPMPC can’t replace online banking powered by a database with chatbot that is 95% correct and 5% hallucinatory
-
AI Progress Claims Not Yet Reliably Solved Despite Advances
By
–
note also that these problems are still not reliably solved despite the allegedly enormous progress since.
-
Do AI Systems Hallucinate? Evaluating Truthfulness and Reliability
By
–
Let’s say you have a system that presents as if it can write biographies, summarize articles, interact w users, generate reports, etc. You can then ask two questions – Does the system invent stuff that isn’t true in the course of carrying out those functions? – Is the system
-
LLMs Hallucinate Inventions While Databases Don’t
By
–
getting something right is, again, not the same as not hallucinating. There are multiple sources of error; whole cloth inventions are different eg from misparsing of instructions. Databases don’t just invent things but LLMs do.
-
AI Honesty Over Arbitrary Problem-Solving Capabilities
By
–
It seemed to me that the interesting claim was not that AIs would start getting arbitrary problems right, but that AIs would in nearly full generality stop making stuff up if they didn't know.