RESEARCH - AI Dynamics

Evaluating Google’s Titan and PaLM Embeddings with MTEB Benchmarks

By

–

06 December 2023 17h21

Here I am trying to figure out how I can get MTEB scores for Titan and PaLM embeddings and doing GPU hour math and I'm not cool enough to get a technical report for a Big Boy Model from Google.

→ View original post on X — @mattlynley

6 December 2023

Gemini Model Card Analysis Delayed by Jury Duty

By

@mattlynley

–

06 December 2023 17h18

Still reading through the model card/report on Gemini (which I did not make the pre-brief cut for this one, alas) so issue will be coming later due to getting called in for jury duty yesterday!

→ View original post on X — @mattlynley

6 December 2023

Google Missed Opportunity Renaming AI Hallucinations

By

@mattlynley

–

06 December 2023 17h15

(Also very much a missed opportunity for Google to rename hallucinations here, which is a terrible name for it.)

→ View original post on X — @mattlynley

6 December 2023

Data Quality Over Quantity: Gemini’s Intentional Uncertainty Handling

By

@mattlynley

–

06 December 2023 17h13

Data quality > data quantity, as we've been talking about since september :). Fun part here, it includes examples where Gemini intentionally throws up its hands (like another model out there). You'd be surprised how useful people think this is. https://
supervised.news/p/ai-in-septem
ber-reasoning-engines
…

→ View original post on X — @mattlynley

6 December 2023

Google’s JAX AI Framework Undergoes First Major Test

By

@mattlynley

–

06 December 2023 17h07

Google's new home-grown AI infra framework JAX getting its first mega-test here.

→ View original post on X — @mattlynley

6 December 2023

Google adopts Meta’s approach training smaller models massive datasets

By

@mattlynley

–

06 December 2023 17h04

And Google now running with Meta's approach of training smaller models with massive data sets. If you're curious about size, the open source RedPajamas open data set is 30T tokens. We can prob safely assume Google has access to much more.

→ View original post on X — @mattlynley

6 December 2023

Google DeepMind Launches Gemini 1.0 with Impressive 90% MMLU

By

@alexandr_wang

–

06 December 2023 17h03

congratulations to @sundarpichai @demishassabis and the whole @GoogleDeepMind team on Gemini 1.0! MMLU of 90% is impressive if the benchmarks hold in use and application of the models, this is pretty groundbreaking.

→ View original post on X — @alexandr_wang

6 December 2023