you know im obligated to respond with this: http://
arxiv.org/abs/1908.04626
@jxmnop
-
arxiv research paper obligation response link shared
By
–
-
Visualizing Transformer Internals: Attribution Graphs and Beyond
By
–
you're talking about the attribution graphs lib i assume? anyway — there is definitely room for more people to work on visualizing the internals of transformers
-
Exploring Alternative Visualizations for Transformer Internal Mechanisms
By
–
surely there are other ways to visualize the internal mechanisms of transformers
-
Why Attention Visualizations Disappeared From Modern AI Models
By
–
when people were working on BERT i always found these types of visualizations compelling. seeing the attention mechanism in action is so cool why are they not popular anymore? do our models have too many layers for us to understand now? or are attention maps just not useful?
-
Open Models Data Gap: Why Algorithms Aren’t Enough
By
–
every new model is a combination of two things, training algorithms and data open-source algorithms are up-to-date. the things that work are usually fairly simple but the data is complex, massive, gatekept, ever-changing. here open models are way behind, and probably hopeless
-
Deep Technical AI Research Talks Should Amplify More Voices
By
–
the early days of the Lex Fridman show featured deep technical talks by great AI researchers, including several who hadn’t spoken publicly much this was excellent, and i would like to see more of it. there are many more people out there who can speak articulately about research.
-
PhD Timeline: From Papers to Autonomous Claude Agents
By
–
timeline of my phd
– year 1: – year 2: write paper on embedding inversion
– year 3: brick conda environment. spend all year fixing
– year 4: spin up 100 separate claude codes with custom prompts for different research problems
– year 5 (soon): check in on my claudes. profit -
Finding the single starting point for runnable group notebooks
By
–
ok but what's the number ONE place to start? isn't there a single grpo notebook that's runnable somewhere
-
Fine-tuning LLM with RL becomes surprisingly easy to implement
By
–
today i finetuned an LLM with RL for the first time. i regret to inform you that it was easy. it only took a few hours to configure. even though this is a custom task and dataset. and it worked, quite well, on the first run
-
Spectrograms and spatial priors in audio baseline models
By
–
we worked with spectrograms. the spatial prior is useful for audio and i was starting from this (very good) baseline model from bytedance: