RESEARCH

Multi-Head Attention: How Output Projection Enables Information Sharing

By

–

12 February 2023 1h19

I haven’t gone back to the paper, but looking at this equation alone: When the heads are independent, W_o introduces correlations among them, allowing for information sharing and a denser representation.

→ View original post on X — @marcusborba

Inside the Heart of ChatGPT’s Darkness: LLM Risks Exposed

By

@marcusborba

–

12 February 2023 1h00

Inside the Heart of #ChatGPT’s Darkness Nightmare on LLM Street https://
garymarcus.substack.com/p/inside-the-h
eart-of-chatgpts-darkness
… by @garymarcus #AI #MachineLearning #DeepLearning @FmFrancoise @Shi4Tech @gvalan @data_nerd #DataScience #BigData #Analytics @ahier @Damien_CABADI @anijov @JAdP @avrohomg @CatherineAdenle

→ View original post on X — @packym

Multiple Alien-Level Threats Emerging Simultaneously

By

@packym

–

12 February 2023 0h23

Cool that there are so many alien-threatening things happening all at once!

→ View original post on X — @jeande_d

Deep Learning and Computational Physics Lecture Notes Shared

By

@jeande_d

–

11 February 2023 22h47

(It's always nice to see people that condense a huge topic like DL into ~80 pages document and share it with the community) Deep Learning and Computational Physics – Lecture Notes: https://
arxiv.org/abs/2301.00942
v1
…

→ View original post on X — @jeande_d

Deep Learning and Computational Physics Lecture Notes from USC

By

@jeande_d

–

11 February 2023 22h47

Deep Learning and Computational Physics – Lecture Notes, University of South California Great & concise notes on various fundamental topics in deep learning. The notes got a nice structure. Starts from the very basics, gradually to some DL architectures. https://
arxiv.org/abs/2301.00942
v1
…

→ View original post on X — @packym

Genius Minds Hide Behind Ordinary Everyday Appearances

By

@packym

–

11 February 2023 22h16

it's discombobulating to look at lifestyle pictures of super geniuses. just goofing around on the beach with a ratty t-shirt looking very normal, and inside their head are the keys to nuclear fusion.

Data Selection and Distribution Functions in Large Language Models

By

–

11 February 2023 19h37

The expectation is replaced by an average over tokens (a few trillion for the largest LMs) so F can be very general. If a human is selecting the y’s among other y’s to create a dataset, then the human is F, hopefully being sensible

Dagger Imitation Learning: Human Feedback for Agent Training

By

–

11 February 2023 19h27

Imitation with Dagger: In counterfactual learning F is typically the identity. The agent acting with policy p(y|x) determines the x’s as in RL, but humans (or other agents) provide corrections in the form of y’s. The new data is used for retraining.

Self-Training: Filtering Functions and Model Ranking Systems

By

–

11 February 2023 19h25

Self-training: F(x,y) is a filtering/ranking function, eg., what we call a reward/return. The input x may be chosen by humans, but the model generates the y’s and F ranks and selects for further rounds of self-training. F can be explicit or implicit (human in the loop as in RLHF)