having spent a little bit of time working on agents, the bottleneck if often not inference speed but interaction speed. Interacting with buttons and browsers and UIs is so inefficient and much slower than say if you had direct API access, and very often you don't.
@soumithchintala
-
GPU Firmware Open-Source Challenges: Vendor IP Rights
By
–
generally, firmware is a lot more difficult to open-source because there might be IP in there that the company might not own (of vendor chips that are on the GPU board, etc.)
-
PyTorch Pay-Per-Use Model Through Advanced Compiler Technology
By
–
I think some day, projects like PyTorch will make users pay only for the overhead they use. Lots of R&D needed to get there: compiler technology, type systems, various other things. Actually, LuaJIT was really cool in that aspect — one of the most brilliant pieces of
-
Growing Availability of Open-Source AI Models Benefits Industry
By
–
+1 the more open-models available at this level, the better
-
Llama3-70B Ranks 5th in LMSys Arena Early Results
By
–
very early LMSys Arena results peg llama3-70B at 5th place (the variance is still pretty high, so it can jump up or down a bit).
This is so exciting.
Can't wait to see how the 405B fares once it is released. https://
chat.lmsys.org/?leaderboard -
Popular LLM Training Frameworks: Hugging Face Transformers Axolotl UnSloth
By
–
in the community: huggingface transformers, axolotl, unsloth, lit-gpt are popular options.
-
Meta Releases Live Preview and Animation Features for Image Generation
By
–
There's another quieter release from @AIatMeta today that's really cool.
* Live Preview: As you type your image prompt, you get a live preview, making iterating for a good image easier.
* Animate: now you can animate images for short bursts -
Memory and Battery Limitations in Laptop Development
By
–
i find memory and battery to be a limiting factor for laptop development. i'm still stuck on an M1 macbook with 16GB memory, and once i start doing computing on the laptop battery depletes too quickly (like on a flight and stuff).
of course i can buy one of those ultra macbooks, -
Gemini’s Published Content Gap: A Tech Industry Observation
By
–
as the semi-serious joke goes: if they published it, its not in gemini