*Sigh.* I knew that meant "principal components analysis" without looking it up. I also knew about the controversy with ICA being patented. Now, what specific implication do you think PCA has for alignment work, exactly?
@esyudkowsky
-
SGD and Transformer Network Depth Rules Matter
By
–
No, I asserted here that SGD was *a* thing. I mention elsewhere that the fact that a classical 100-layer transformer network obeys its own "100-step rule" is another thing that matters (and also is not inaccessibly deep mathematics).
-
Dataset Filtering for Large Language Model Training Runs
By
–
Possibly yes. More broadly, if we're going to be doing more large training runs at all, I'd guess we should start filtering the datasets soon (presumably using a previous-generation LLM finetuned for that?). It's hard to finetune out a cognition once it's learned by the base.
-
AGI Alignment Knowledge and LLM Developer Expertise Skepticism
By
–
Maybe I shouldn't, but one last shot at explaining my position on AGI gatekeeping. I'm not saying that I know everything known to the high-status inventor or engineer of the largest LLM. I'm saying that I'm dubious that they have secret knowledge deeply relevant to *alignment*,
-
Gatekeeping the Gatekeepers of Deep Learning
By
–
I am not trying to gatekeep modern deep learning. The relevant truths of DL are open to anyone who's even slightly good at math! I'm trying to gatekeep the gatekeepers who shouldn't be allowed to gatekeep something with such a low gate.
-
Finetuning Does Not Defeat Original Argument Against Imitative GPTs
By
–
TOC:
– (1) The existence of finetuning does not defeat the point of the original tweet
– (2) You propagated other clear misunderstandings in your further QTed dunks. (1) My original tweet argues against seeing GPTs as being imitative. In particular, if you read the following -
Democratizing AI Knowledge: Breaking Down Gatekeeper Barriers
By
–
Not gatekeeping AI (or rather DL), which is surprisingly friendly in terms of how simple the math is that will let you get a grasp on the macro-relevant issues; gatekeeping the gatekeepers, who ought not to be allowed to gatekeep something so easy.
-
Knowledge of Positional Embeddings and Attention Mechanism Improvements
By
–
Like I said, I already knew about positional embeddings being trained rather than being straight from the 2018 paper and had heard of axial transformers and various other attempts to defeat quadratic attention. (I think I can predict your next gatekeep, but go ahead and do it.)
-
Eliezer Responds to Criticism About His OpenAI Position
By
–
This is frankly an odd take from you, Michael. In what possible world of your imagination would I have not been immediately horrified by OpenAI? What kind of fantastical alter-Eliezer have you been constructing in your mind?
-
Eliezer’s Horror at OpenAI’s Early Actions and MIRI’s Strategic Silence
By
–
I was horrified at the time. Couldn't say anything because the much tinier MIRI could have been easily targeted and squashed if it'd openly opposed OpenAI and its founders/funders at the time, but I was horrified at the time. Unlikely to be confabulating the memory because I