People overestimate the coherence and intention of ChatGPT’s apparent “policies”. It doesn’t know how OpenAI would like it to behave in every scenario, so it guesses, imperfectly. E.g. no human would write the “rule” below — it’s just how the fine-tune cake came out of the oven:
LLMS
-
@peterwildeford notes GPT-3 poor at BIG-bench ASCII word recognition
By
–
As @peterwildeford points out a version of this task is in BIG-bench and GPT-3 is really bad at it: https://
github.com/google/BIG-ben
ch/tree/main/bigbench/benchmark_tasks/ascii_word_recognition
… -
Enable ‘Show probabilities’ then hover over tokens
By
–
“Show probabilities” in the options, then generate and hover over specific tokens.
-
ChatGPT critical review and technology governance for 2023
By
–
Un repaso a ChatGPT, una mirada crítica a la hipérbole tecnológica de 2022 y una palabra clave para 2023: #Gobernanza. En @disruptores_inn, celebrando 5 años de #Descartes con 50 artículos https://
elespanol.com/invertia/disru
ptores-innovadores/opinion/20230115/avance-tecnologico-no-ciegue/733556651_13.html
… -
AI says ‘mountain’ frequently, even for ASCII smiley face art
By
–
It says “mountain” a lot though. I gave it ASCII art of a smiley face and it said “mountain” too. I suspect it has some intuition about shapes being vaguely square vs. triangular etc. but mostly I think it’s good at guessing what typical subjects are.
-
ChatGPT Invention Bias on Global Warming Truth
By
–
ChatGPT will make anything up, but on global warming it knows Only One Truth.
-
What Happens When AI Has Read Everything Online?
By
–
What Happens When #AI Has Read Everything? https://
theatlantic.com/technology/arc
hive/2023/01/artificial-intelligence-ai-chatgpt-dall-e-2-learning/672754/
… @TheAtlantic #MachineLearning #BigData #Analytics @Shi4Tech @CatherineAdenle #100DaysofCode #womenwhocode #serverless #ML @anijov #IoT @FmFrancoise @BIScorecard #DataScience #DeepLearning #DigitalTransformation -
Computers as Syntactic Machines and LLMs’ Semantic Progress
By
–
Computers are syntactic machines. Large language models are a small step toward semantics.
-
GPT-3 understands ASCII art but not rasterized images
By
–
It definitely can understand small, cartoon-style ASCII art (e.g. it knows ">" is a fish and "(o_o)" is a face) but it isn't good at rasterized images. E.g. GPT-3 could do this back in August:
-
GPT-3 identifies ASCII art MNIST as ‘8’ 57%, ‘4’ absent in top-5
By
–
There's a thread going around claiming ChatGPT / GPT‑3 can recognize subjects (e.g. "a bird", "a person") in images rasterized as ASCII art. I'm skeptical. E.g., GPT‑3 identifies this ASCII-art MNIST digit as "8" 57% of the time and "4" doesn't appear in its top-5 choices: