went on a podcast to talk about assorted topics:
– life
– research
– AI
– grad school
– AI grad school
and a lot of other things give it a listen!
@jxmnop
-
Podcast Discussion on AI Research Graduate School and Life
By
–
-
Neural Networks Learn Regardless of Weight Initialization and Hidden Patterns
By
–
it's crazy to me that neural networks learn ~arbitrarily well regardless of initialization. you can even embed patterns in the weights and learning works fine you could encode an image of your face into the layers of a language model and no one would ever know
-
Open-Source AI Models Struggle to Match Closed Competitors
By
–
had the depressing realization that since chatGPT came out, there has *never* been a moment when the best model was open-source the open models just never quite manage to catch up to closed ones before the the next one drops
-
Transformers as Universal Computation: Text Models Work for Images
By
–
this paper blew my mind when i read it four years ago. it blew my mind again when i reread it this morning basic finding is that transformers pretrained on TEXT are also useful for IMAGES (and proteins, and numerical operations)…. i think the term "Universal Computation
-
Memory and reasoning in AI models: dependencies and relationships
By
–
why would giving models memory depend on reasoning?
-
AI Labs Should Prioritize Memory Over Extended Reasoning
By
–
seems big AI labs are hyperfixating on reasoning when they should focus on *memory* instead normal people won't use models that can think for hours to solve hard math problems people want models that learn over time, remember details, adapt and interact like a person would
-
Current AI Reasoning and Memory Capabilities Still Insufficient
By
–
i don't think either one is good enough already (reasoning or memory)
-
RNNs Learn to Adapt at Test Time with Expressive Hidden States
By
–
[ii] Learning to (Learn at Test Time): RNNs with Expressive Hidden States
-
Learning to reason with LLMs: o1 blog post analysis
By
–
[i] Learning to reason with LLMs (the o1 blog post)