If you want something about kqv matrices, you're asking for an explanation on the wrong level of abstraction; if the problem was specific to kqv matrices we'd advocate "stop using transformer layers" not "shut down AGI research".
RESEARCH
-
Utility Functions and Existential Risk from Instrumental Convergence
By
–
https://
arbital.com/p/instrumental
_convergence/
… but I'm not sure what that buys you if "pick any simple measure on utility functions, preimage them through a reasonable environmental model onto actions, most utility functions kill humanity as a side effect" doesn't already do it. -
Neural Networks: Billions of Inscrutable Numbers We Cannot Understand
By
–
Nope! Neural nets are built by repeatedly poking a table of hundreds of billions of inscrutable numbers until they start doing what the builders want. We understand the thing that does the poking, but not what the hundreds of billions of numbers mean.
-
Preference Functions and Alignment: Optimizing for Human Flourishing
By
–
If you don't see why most preference functions that don't specifically have an attainable optimum around people living happily ever after, do something else instead of that, I'm not sure what particular inscrutable properties of a kqv layer are going to be persuasive?
-
Instrumental Convergence Denial Prevents Advanced AI Safety Discussion
By
–
Yann is currently at the stage of denying instrumental convergence, so there's no point in bringing in anything more complicated from List of Lethalities.
-
Debating AI Safety: The Unbridgeable Gap in Risk Perception
By
–
If someone performs that they can't see any difference between building a poorly understood superhuman intelligence, and a 20th-century newspaper article inveighing against coffee, they are beyond the reach of debate. I can only go to the general public and say, "This is their
-
GPT-4 Turbo outperforms GPT-4: surprising benchmark results analyzed
By
–
surprised to see so many people excited to see google sitting in second place on a leaderboard also, the obvious question here is why is GPT-4 turbo beating GPT-4 on this benchmark? i thought turbo was intended to be faster but slightly dumber
-
AI Safety Gaps and Recursive Self-Improvement Innovation
By
–
Good read on AI safety, how it’s falling short, and one possible approach to keep up the rate of innovation. A few key takeaways: “the full scope of the potential new capabilities that could be enabled by recursive self-improvement is not known.” “combinatorial innovation,”
-
Input Embeddings Keys Queries Values Attention Mechanism
By
–
Now here's a broader picture of how input embeddings are combined with Keys, Queries & Values to obtain the actual attention scores. After acquiring keys, queries, and values, we merge them to create a new set of context-aware embeddings. Check this out
-
Deep Learning Advances Medical Pathology Diagnosis with MATLAB
By
–
Improving Deep Learning-Assisted Pathology with @MATLAB Explore the advancement of medical diagnosis technology http://
spr.ly/6018TfFBi #DeepLearning