INNOVATION

JAX Omission Raises Questions About AI Training Technology Strategy

By

–

05 May 2023 18h25

There’s no mention of JAX. That was a little weird as they own an extremely powerful next-generation training technology in the same way they owned TensorFlow at first.

Google’s Strategic Missteps in AI and Open Source Innovation

By

–

05 May 2023 18h25

Google keeps falling asleep at the wheel here. It didn’t productize MapReduce, allowing Hadoop to happen. It let PyTorch blossom and take over the DS/DL community. And now it has basically let competitors run off with the Transformers technology.

GPT-4 Architecture: Parameter Count and Competitive Moat Analysis

By

–

05 May 2023 18h25

OpenAI is mentioned in here as having no moat, but OpenAI was also intentionally silent about the parameter count of GPT-4. I know some people have been flinging around a 1T parameter number for GPT-4 but it’s very possible that it isn’t a single 1T model and instead 1T total.

Meta’s FAIR Outpaces Google in Open Source AI Leadership

By

–

05 May 2023 18h24

Meta comes out looking so good from this. Meta’s AI division FAIR has consistently outmaneuvered Google in the open source deep learning community, starting with PyTorch and now with LLaMA.

Custom Language Models Spawn New Generation of AI Startups

By

–

05 May 2023 18h24

An entire generation of startups are going to be born and grow on the back of these technologies because they are so lightweight and cheap to make custom implementations. I have yet to talk to a company deploying language modeling that is not playing around with custom models.

Open Source LLaMA Models Achieving Foundation Model Performance

By

–

05 May 2023 18h24

What is new is just how many times this has been applied to LLaMA and other models to squeeze near-equivalent foundation model performance out of a small number of parameters. But that’s basically what happens when the cat comes out of the bag in open source.

LoRA and Quantization: Google Shifts from Big Numbers Approach

By

–

05 May 2023 18h23

One of my big takeaways is just how important LoRA—basically a way to simplify and reduce the cost re-training models—and quantization for more compact models is here. Google has traditionally taken the Big Numbers approach to model development so it’s at minimum notable.

→ View original post on X — @thegautamkamath

Geoff Hinton: The Old Dude Who Created AI

By

@thegautamkamath

–

05 May 2023 18h16

With all due respect, Geoff Hinton should change his Twitter bio to
"The old dude who created AI" – @SnoopDogg https://t.co/o4dffJNwRu
— Gautam Kamath (@thegautamkamath) 5 mai 2023

With all due respect, Geoff Hinton should change his Twitter bio to
"The old dude who created AI" – @SnoopDogg

→ View original post on X — @mit_csail

Company Age Comparison: Tech Giants from OpenAI to Nokia

By

@mit_csail

–

05 May 2023 18h12

Companies by age: OpenAI: 8 years Twitter: 17 years Facebook: 19 years Tesla: 20 years Google: 25 years Amazon: 29 years Apple: 47 years Microsoft: 48 years Sony: 77 years IBM: 112 years Nintendo: 134 years Nokia: 158 years v/
@spectatorindex