a 3.25B params quantized gemini running locally in coming Google Chrome with less than 100ms latency while using less than 2GB of ram

that's less ram usage than many of my current Chrome page already use (my slack is using 4.8GB as I type this)

no doubt LLMs will be integrated… https://t.co/Muy8o4qPVV

— Thomas Wolf (@Thom_Wolf) 24 juin 2024

a 3.25B params quantized gemini running locally in coming Google Chrome with less than 100ms latency while using less than 2GB of ram that's less ram usage than many of my current Chrome page already use (my slack is using 4.8GB as I type this) no doubt LLMs will be integrated

→ View original post on X — @thom_wolf,