fun research idea: Latent chain-of-thought / Latent scratchpad it's well-known that language models perform better when they generate intermediate reasoning tokens through some sort of 'scratchpad'. but there's no reason scratchpad tokens need to be human-readable. in fact,
@jxmnop
-
Google’s AI Model: Closer Than Expected to AGI
By
–
eh I don't think there's anything laughable about it; it's quite a good model; just a bit further from AGI than GPT-4 I guess. wouldn't be surprised if Google found a way to serve it much more cheaply though.
-
LMSYS Chatbot Arena Head-to-Head Comparison Discussion
By
–
this is a good point, for the head-to-head lmsys chatbot arena you are right!
-
GPT-4 Outperforms Gemini: Google’s AI Strategy Misstep
By
–
Seen a lot of evidence that GPT-4 crushes Gemini on all the head-to-head LM benchmarks. here's my theory about what went wrong: – chatGPT released
– google execs freak out
– google consolidates (integrates Deepmind, Brain)
– google assembles a giant team to build a single giant -
Machine Learning Research: Room for Niche Ideas in High-Dimensional Space
By
–
i used to find this kind of thing intimidating – ML is so popular, competitive, etc
— dr. jack morris (@jxmnop) 16 décembre 2023
but I realized how different every paper is. each presenter here traveled down one single esoteric rabbit hole
high-dimensional spaces are so sparse; there's still plenty of room for your ideas https://t.co/U7jR3OefBJi used to find this kind of thing intimidating – ML is so popular, competitive, etc but I realized how different every paper is. each presenter here traveled down one single esoteric rabbit hole high-dimensional spaces are so sparse; there's still plenty of room for your ideas
-
Beyond Language: Why GPT-4 Needs Training Data
By
–
long story short, my task can't be completed by GPT-4; the data isn't only language, it includes continuous vectors, so you really need training data for this
-
Iterative Translation Optimization Using BLEU Score Maximization
By
–
i was essentially doing translation 1000 times, computing the BLEU scores of the translations, and returning the max BLEU score …
-
Importance of bug verification before publishing code
By
–
thank goodness I waited to send my tweet until i confirmed the presence of a bug your mentor would be so proud
-
PhD Student’s Journey: From Idea to Breakthrough Research Results
By
–
the saga of a phd student > last week, have ground breaking research idea (so i think, anyway)
> two days ago, write sloppy prototype code for idea > yesterday, run experiments, collect results
> wow, results surprisingly good
> this is so cool, that was easy
> i love grad -
Grateful for Computer-Based Scientific Research
By
–
it's times like these i feel thankful to do my science on the computer