Super proud of our researchers for winning the MAG240M Track of the OGB-LSC competition at #NeurIPS2022! They used PGL and PaddlePaddle to accelerate GNN training on GPUs, reaching SOTA in only 1 hr (40 hrs before). Check out their code: https://
github.com/PaddlePaddle/P
GL/tree/main/examples/NeurIPS2022-OGB-Challenge/MAG240M
…
RESEARCH
-
Researchers Win OGB-LSC Competition with Accelerated GNN Training
By
–
-
ChatGPT tweets liked by Elon Musk curated
By
–
Filter down to content you’re curious about:
"tweets about chatGPT liked by Elon Musk" https://
perplexity.ai/sql?uuid=0a675
88c-2ecb-4696-88cf-4aa321cf3a96
… -
Discover Top Astrophysicists to Follow Online
By
–
Find new users to follow:
"top astrophysicists" https://
perplexity.ai/sql?uuid=1492d
123-dc05-4ea3-bedf-9aea8a918749
… -
ERNIE-Code: Unified Multilingual Code-to-Text Model
By
–
Introducing ERNIE-Code, a unified pre-trained language model for 116 natural languages and 6 programming languages. ERNIE-Code outperforms previous models for multilingual code-to-text, text-to-code, code-to-code, and text-to-text generation. Paper: https://
arxiv.org/abs/2212.06742 -
AI Technology: Powerful but Not Human Intelligence
By
–
It has been “fed” thousands of documents up to 2021 so that you can ask about a given topic and be given a simple answer. Yes it can write in poetic form or write a one pager on climate change but it is #nothuman
-
Public and Private AI Training: Different Accuracy Standards
By
–
Also, my personal opinion is that both public pretraining (with certain constraints/limitations) as well as private training from scratch both have roles to play in the space. They of course can't be held to the same accuracy standard though.
-
How GPT Obtains Its Ability: Tracing Emergent Abilities
By
–
It’s from here: https://
yaofu.notion.site/How-does-GPT-O
btain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
… -
Public vs Proprietary Datasets in AI Model Training
By
–
Thanks Alex. I like the papers that do this, but I also have some concern when this is done on a dataset that is proprietary and only Google has access (JFT). I would like to see a version that is pretrained on LAION. This still has privacy issues, but is at least all public.
-
Control Text Generation Model Creativity: Temperature Top-k Top-p
By
–
To recap, you can adjust 3 parameters to control how creative a text generation model’s output are:
1. Temperature
2. Top-k
3. Top-p Lower values = more predictable.
Higher values = more creative. More on this blog: https://
txt.cohere.ai/generative-ai-
part-1/
… -
Top-k Sampling Control in AI Model Generation
By
–
Let’s take the same example of startup ideas brainstorming. Now, set the top-k to 1. This means the model can choose from exactly 1 option. So, no matter how many times you generate, or what temperature value you set, the output will be the same every time, as below.