The tech breakthroughs of 2022 included #ChatGPT as well as the @DeepMind protein-folding algorithm and a step in nuclear #fusion. #Innovation isn’t dead.
@angusloten
here is to good things in 2023
RESEARCH
-
2022 Tech Breakthroughs: ChatGPT, Protein Folding, Nuclear Fusion
By
–
-
OpenAI Whisper Speech Recognition Model Added to Hugging Face
By
–
In October, Whisper, a state-of-the-art audio model from @OpenAI for speech recognition was added to the library. https://
huggingface.co/docs/transform
ers/model_doc/whisper
… -
LayoutLM v3 Multimodal Model Added for Document Analysis
By
–
LayoutLM v3 (also from @MSFTResearch
) was added to the library in June. It is a multimodal model combining vision and text for document analysis. -
Meta AI Open-Sources OPT Language Model Series
By
–
OPT was open-sourced by @MetaAI and added in the library last May. It is a series of open-sourced large causal language models similar in sizes to GPT3.
-
BLOOM Multilingual Language Model Released by BigScience
By
–
BLOOM was then released in June by @BigscienceW
. It is a similar series of model, though trained on 46 languages. https://
huggingface.co/docs/transform
ers/model_doc/bloom
… -
Doubling AI Model Architectures: New Audio, Vision and Multimodal Models
By
–
We doubled the number of architectures (89 to 167) with new models in audio, text, vision, multiple modalities or even time seriesand protein folding Here are a few highlights in the most used of those new models
-
Swin Transformer Vision Model for Image Classification Detection
By
–
Swin Transformer is a vision model from @MSFTResearch added back in January, which can be used as backbone for a variety of tasks such as image classification, object detection or semantic segmentation. https://
huggingface.co/docs/transform
ers/model_doc/swin
… -
Transformers Library Triples Users, Reaches 1M Weekly Active
By
–
It's been an exciting year for Transformers. We tripled the number of weekly active users over 2022, with over 1M users most weeks now and 300k daily pip installs on average
-
Model Evaluation Trade-offs: Epochs versus Second-Order Methods
By
–
Sure. But you only get to do 60,000*20 model evals in total either way! (i.e if the 2nd order method needs twice as many evals, you only get 10 epochs)
-
GPT-3 Synthesis of arXiv Preprint with BERT-embedded Hopfield Networks on Ethereum Blockchain
By
–
GPT-3 zero-shot synthesis of an arXiv pre-print in LaTeX notation on distributed manifold in-painting using BERT-embedded Hopfield networks on the Ethereum blockchain. Model text-davinci-003, temperature 0.7. Playground link: https://
beta.openai.com/playground/p/m
z75vuUjGsEusUyPheyEdcSZ?model=text-davinci-003
…