I think there’s not enough training data naturally on the internet of spelling tasks compared to the difficulty of the task for the LLM, due to how text is chopped up into sequences of text chunks (tokens), all of which are unique / distinct. I have a whole video on Tokenization.
LLMS
-
Neural Scaling Laws Workshop Highlights from ICML 2024
By
–
Highlights from #ICML 2024! We had an incredible time at the 7th Workshop on Neural Scaling Laws: Scaling, Transfer & Multilingual Models, held earlier this week at ICML in Vienna. This joint event, co-organized with Irina Rish and CERC-AAI, brought together brilliant
-
Learning Orthogonalized Matrices vs Computing Refusal Directions
By
–
I'd say it's different because you learn the orthogonalized matrix with OFT while you compute (you don't learn) your refusal direction here
-
Model Weight Redaction Techniques in AI Systems
By
–
Yes it's kind of redacting the model's weights in a way
-
Google Integrates AlphaProof and AlphaGeometry into Gemini Models
By
–
We have long pioneered the use of these types of neuro-symbolic systems starting with AlphaGo in 2016, through to AlphaZero. We'll be bringing all the goodness of AlphaProof and AlphaGeometry 2 to our mainstream #Gemini models very soon. Watch this space!
-
User reports functional issues with Gemini AI web integration and citations
By
–
Related content on Gemini seems to be only working on the web and often not quite as expected @GeminiApp – Empty previews are shown as the URL is being composed as "google com / testingcatalog com" – For some queries, it is working but showing only one citation instead of
-
OpenAI clarifies OAI-SearchBot functionality and data usage policy
By
–
Bing but also with an LLM on top. "OAI-SearchBot is for search. OAI-SearchBot is used to link to and surface websites in search results in the SearchGPT prototype. It is not used to crawl content to train OpenAI’s generative AI foundation models." https://
platform.openai.com/docs/bots -
Language reasoning system advances combinatorics
By
–
We love combinatorics! I shared some info here on our language reasoning system
-
Gemini Language Reasoning System Tackles IMO Mathematics Problems
By
–
Last but not least, we experimented with a Gemini-based language reasoning system that showed great promise at this year’s IMO problems. This system doesn’t require the problems to be translated into a formal language and can directly generate and verify solutions in
-
Privacy and Operational Details for SearchGPT Prototype
By
–
More statements from the onboarding page
– It is a WIP prototype
– It may collect your location info
– Your search queries might be reviewed – Account history for SearchGPT is separate from ChatGPT