GPT-3 has been trained on 45 TB of text data from different categories: ⬩Common Crawl (8 years of raw web page crawling) ⬩WebText (The text of Reddit posts with 3+ upvotes) ⬩Books (The internet-based books corpora) ⬩Wikipedia Data is then "weighed" as such:
RESEARCH
-
CICERO by Meta AI: Strategic Dialogue Generation System
By
–
#CICERObyMetaAI generates strategic, free-form dialogue by: Predicting the moves other players are likely to make Creating a plan based on those predictions Generating messages Filtering the final messages for clarity and value Read more on our blog
-
VectorFusion Generates Vector Graphics Without Large Datasets
By
–
VectorFusion can produce vector graphics without needing access to large datasets of captioned vector images. https://
ajayj.com/vectorfusion -
MagicVideo: Text-to-Video Generation Framework Using Diffusion
By
–
MagicVideo is an efficient text-to-video generation framework based on latent diffusion models that can produce photorealistic video clips from text descriptions.https://t.co/SUEk3csiB7 pic.twitter.com/MqVcOIwNY0
— KREA AI (@krea_ai) 23 novembre 2022MagicVideo is an efficient text-to-video generation framework based on latent diffusion models that can produce photorealistic video clips from text descriptions. https://
magicvideo.github.io -
Large Language Models Should Be Called Infrastructure Models
By
–
A better name for large language models is infrastructure models.
-
ACM Gordon Bell Prize Awarded for HPC COVID-19 Research
By
–
Congratulations to our Solutions Engineers, Claire Zhang and Cindy Catherine Orozco Bohorquez, who recently received their ACM Gordon Bell Special Prize for HPC-Based COVID-19 Research awards! Read the paper here: https://
hubs.li/Q01s_n7N0 -
Old NLP Experts Underestimate Language Models’ Capabilities
By
–
Some old-time NLPers can be closed minded about the "range of tasks" that LMs can do. They are still thinking about LMs through BERT, etc. I rarely see this issue with new-joiners to NLP.
-
David Ash Supports Waterloo Math Program as Putnam Fellow
By
–
Very cool support of @WaterlooMath from David Ash. David is a Waterloo alum, and three-time Putnam Fellow. Only 31 people have ever been a Putnam Fellow >= 3 times.
-
Hugging Face Datasets Library Appreciation and Documentation
By
–
This is a @huggingface Datasets appreciation tweet! https://
huggingface.co/docs/datasets/
index
… -
NeurIPS 2022 Announces Outstanding Papers and Awards
By
–
NeurIPS 2022 Announces Its Outstanding Main Track Papers, Outstanding Dataset & Benchmark Papers, and Test of Time Award