RESEARCH

→ View original post on X — @rachelmetz

Technological Advances Behind Large Language Models Take Years

By

@rachelmetz

–

12 June 2023 22h26

Technological advances don’t happen overnight; the technology that helped make it possible for us to have large language models today (like that which underpins ChatGPT) came out six years ago. https://t.co/Z12ZglKNPN
— Rachel Metz (@rachelmetz) 12 juin 2023

Technological advances don’t happen overnight; the technology that helped make it possible for us to have large language models today (like that which underpins ChatGPT) came out six years ago.

→ View original post on X — @huggingface

FAccT2023 Conference: AI Ethics and Fairness Research Presentations

By

@huggingface

–

12 June 2023 22h08

We have a lot of cool work @FAccTConference #FAccT2023 this week! Check out tutorials, workshops, and paper presentations from @ezi_ozoani @NimaBoscarino @GiadaPistilli @IreneSolaiman @YJernite @SashaMTL @natolambert @nazneenrajani @Carlos_MFerr and @mmitchell_ai below!

Open Source Model Evaluation Results Exceed Public Competition

By

–

12 June 2023 20h26

thanks for running the evals. indeed the numbers look great (and promising). yeah i think it's scalable and efficient. like your tweet suggested it's a pity there's no t5 or ul2 model >40B. otherwise i think it would beat every public open source model right now.

→ View original post on X — @jeande_d

Advanced Computer Vision Course at UCF Center for Research

By

@jeande_d

–

12 June 2023 19h53

Advanced Computer Vision – UCF Center for Research in Computer Vision, Spring 2023 This is an excellent course on advanced computer vision. Covers recent developments in computer vision, topics like image generation and vision-language learning. I like that the course go

PaLI-X Combines ViT-22B Vision and Multilingual UL2-32B

By

–

12 June 2023 19h49

Check out PaLI-X that combines ViT-22B and multilingual UL2-32B!

Science Over Hype: Debunking AI Claims

By

–

12 June 2023 19h40

this is not about promotion. it's about science and hype busting.

Encoder-Decoder vs Decoder-Only Models: T5 Paper Guide

By

–

12 June 2023 19h39

Additional reading for people confused about encoder-decoder models vs decoder-only models: most wisdom is also from the 2019 T5 paper. it's the bible for NLP literally.

→ View original post on X — @rasbt

Attention Mechanism Origins Traced Back to 1991 Schmidhuber

By

@rasbt

–

12 June 2023 19h12

Oh, and sorry, @DrJimFan
, even the 2014 paper is not the original attention paper. It goes actually back to 1991 (via @SchmidhuberAI
) in https://
semanticscholar.org/paper/Learning
-to-Control-Fast-Weight-Memories-An-to-Schmidhuber/bc22e87a26d020215afe91c751e5bdaddd8e4922
… I've summarized it here: https://
magazine.sebastianraschka.com/p/why-the-orig
inal-transformer-figure
…