LLMS

→ View original post on X — @rowancheung

Google Unveils Gemini 1.5 with Groundbreaking 1M Token Context

By

@rowancheung

–

16 February 2024 3h19

Google just revealed an upgraded Gemini 1.5 model—featuring a groundbreaking 1M token context window (up to 10M in testing).

The LLM can now process considerably more data than competitors and enables (previously unimaginable) capabilities. https://t.co/AIdn9OoUgF
— Rowan Cheung (@rowancheung) 16 février 2024

Google just revealed an upgraded Gemini 1.5 model—featuring a groundbreaking 1M token context window (up to 10M in testing). The LLM can now process considerably more data than competitors and enables (previously unimaginable) capabilities.

→ View original post on X — @rowancheung

Major AI Breakthroughs: OpenAI, Google, Meta Dominate News

By

@rowancheung

–

16 February 2024 3h19

Historic day in AI. First, OpenAI changed the AI video world and Google made enormous progress in LLM capabilities. Then we saw huge developments from Meta, Slack, CodeSignal, X/Grok, Microsoft, University of Michigan, LangChain, and Magic. Here's EVERYTHING you need to know:

→ View original post on X — @mreflow

Apple Develops Text-to-Image Models, Rumored Siri LLM Overhaul

By

@mreflow

–

16 February 2024 1h35

Apple has some interesting text-to-image and image-to-image models they've been working on. Rumor is we'll be getting an overhauled Siri with an Apple-built LLM announcement at WWDC. But I don't have concrete evidence of that yet.

→ View original post on X — @montreal_ai

Rethinking Machine Unlearning for Large Language Models

By

@montreal_ai

–

16 February 2024 1h16

Rethinking Machine Unlearning for Large Language Models Liu et al.: https://
arxiv.org/abs/2402.08787 #ArtificialIntelligence #DeepLearning #LargeLanguageModels

VQVAE Stack Architecture Differs Fundamentally from Transformers

By

–

16 February 2024 1h12

also a “VQVAE stack” is not a transformer! it would not look like this haha

Fine-tuning GPT2 with additional layers avoids pretraining scratch

By

–

16 February 2024 1h09

he’s pointing out we can just add the two layers I drew here and train those starting from pretrained GPT2 or something; don’t have to pretrain from scratch

Discrete Tokens Recover AI Model Performance Effectively

By

–

16 February 2024 1h08

if you use enough discrete tokens you could def recover performance eventually, unless you’re doin somethin wrong

Multi-token sparsity in discrete token activations

By

–

16 February 2024 1h07

yeah it’s similar— I think you could call it multi-token sparsity or something like that? I mean obviously the activations have to be sparse in this case but we can use as many discrete tokens as we want (unlike in the projects you’re mentioning)