Excited to find @Tim_Roughgarden
's Algorithms Illuminated in my mailbox! Took a quick look and seems like an excellent book for learning from (as opposed to a reference manual…). I like the intuitions, exercises, and conversational style. Will try it next time I'm teaching!
MACHINE LEARNING
-
Tim Roughgarden’s Algorithms Illuminated Book Review
By
–
-
Optimizer’s Imprint on Model Basis-Dependence: Research Evidence
By
–
While further research is needed to be totally conclusive, we believe we provide good evidence for this hypothesis. If this is the case, the basis-dependence we see in models is an imprint of the optimizer, rather than an intrinsic feature of the model or loss landscape.
-
Investigating Model Divergence: Adam Optimizer as Primary Cause
By
–
In this research note, we explore a few different theories for this divergence. We looked at layer normalization and floating-point rounding as potential causes, and we were able to rule them out. This leaves the Adam optimizer as the most likely culprit.
-
Residual Stream Theory: Reconsidering Privileged Basis and Large Coordinates
By
–
In prior work, we've claimed that the residual stream has no "privileged basis" and individual coordinates should have no special significance. The presence of persistently-large coordinates suggested our theory might miss something.
-
Transformer Residual Stream Dimension Analysis and Privileged Basis Discovery
By
–
Recent work has observed that some dimensions in the Transformer residual stream consistently have values that are 20x larger than others. Today we're releasing a short technical note exploring this phenomenon and trying to identify the cause. https://
transformer-circuits.pub/2023/privilege
d-basis/index.html
… -
Praise for Topic Modeling Thread and Blog Post
By
–
That's a well-articulated thread on Topic Modeling. BTW the blog is also excellent. Great work
-
Twitter Spaces on Semantic Search with Industry Experts
By
–
Our first ever Twitter Spaces featuring @JayAlammar @Nils_Reimers @bobvanluijt and @svpino is happening today at 12 PM ET! Learn more about Semantic Search here: https://
x.com/CohereAI/statu
s/1633830290438356995
… Have a question for the panellists? Drop your questions in the thread
[1/3] -
China’s AI Ship Designer Completes Year of Work in 24 Hours
By
–
China’s ‘#AI Ship Designer’ Works At Unprecedented Speed; Performed A Year’s Work Only In 24 Hours!
by Tanmay Kadam @THEEURASIATIMES Go to: https://
buff.ly/3ToD9P6 #AI #BigData #ArtificialIntelligence #ML #MI #DataScience cc: @wil_bielert @patrickgunz_ch @terenceleungsf -
K-Means vs K-Means++: Centroid Initialization Differences
By
–
The drawback of k means is how it initializes the centroid compared to the initialization of K means++
The difference is Perfectly explained in this thread Akshay. Great work -
New Snake Species Identification Competition Alert
By
–
NEW COMPETITION ALERT Can you build a robust system for identifying species of snakes from photographs?!! Join this super exciting, one of a kind competition in which we use multiple metrics to rate the performance of your model in a single LB! https://
hf.co/spaces/competi
tions/SnakeCLEF2023
…