Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models https://
bit.ly/3Hhs4LK 4/6
SAFETY
-
Memorization Without Overfitting in Large Language Models Training
By
–
-
Writing prayers to study AI mistakes for improved reliability
By
–
I write a lot of what I call “prayers” — long prompts the are unlikely to work at all. I assume the thing is as smart as a human and then study its mistakes. I then search for variant instructions that improve reliability.
-
ChatGPT refuses to acknowledge the mistake
By
–
I've been struggling with #ChatGPT for 15 minutes trying to explain it's wrong lol, but it just won't get it. It keeps telling me I can use an argument in a function, but that's not true. Even the most powerful AI conversation model on the planet doesn't read the docs lol.
-
Technology innovation making disability support more practical
By
–
Difficult to make it more practical to help people with disabilities!
— Pascal Bornet (@pascal_bornet) 6 décembre 2022
Credit: A. Thobellem#techforgood #innovation pic.twitter.com/isMs0uSYCPDifficult to make it more practical to help people with disabilities! Credit: A. Thobellem
#techforgood #innovation -
Robust Artificial Intelligence: Four Steps Towards AGI
By
–
"absolutely brilliant" —Nobel Laureate Danny Kahneman The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence @GaryMarcus : https://
arxiv.org/abs/2002.06177 #AGI #AIDebate #AGIDebate -
Strategic Advantage Through Secrecy in Arms Competition
By
–
The best way to win an arms race is to keep your superiority secret.
-
ChatGPT removes similar strings, suggesting hidden control codes
By
–
The fact that ChatGPT removes not just this string, but any string formatted similarly, suggests there exist other control codes by ChatGPT, e.g. to mark the boundaries between user input and prior model output within the prompt context.
-
Multimodal Learning for Objective Representation
By
–
To address these issues, it is necessary to learn a vast amount of multimodal information (sound, image, text, etc.) using A LOT of data, in the hope of achieving the most objective representation of the world possible—something that is nearly impossible, yet still feasible with an 80-20 rule.