Yes that's fair, in big O terms the separate QKV should be faster
CODE
-
Simplifying Transformer Blocks: Key Research Paper
By
–
Yes. There was a nice paper on that, which looked into some other modifications: Simplifying Transformer Blocks, https://
arxiv.org/abs/2311.01906 -
Drop-in replacement code for modified attention mechanism available
By
–
Yup, the reader posted the code for a drop-in replacement for the modified attention mechanism. Just copy and paste it into the Chapter 5 code, which is available in the GitHub repo linked above.
-
Merging Query and Key Weight Matrices in LLM Training
By
–
Can we merge the query and key weight matrices in an LLM into a single covariance matrix and still train effectively? Here are some promising early results from a reader: https://
github.com/rasbt/LLMs-fro
m-scratch/discussions/517
…
Anyone else familiar with projects that tried this? -
MongoDB 2025: ACID Compliance and Advanced Data Handling
By
–
In 2015 MongoDB was the easiest database for developers to use. That's still true in 2025, but every other myth you may be clinging to is gone: it's ACID compliant, does joins (but offers superior ways to handle relational data), etc. Have a look.
-
Meta’s Automated Compliance Hardening Tool Uses LLM for Testing
By
–
Our Automated Compliance Hardening (ACH) tool hardens platforms against regressions with LLM-based test generation. More on how this works and why it matters at Meta https://
go.fb.me/jyaab2 -
DeepSeek-R1 671B Enhances Coding Performance on BlackBox Cloud
By
–
🐋 DeepSeek-R1 671B is on our Cloud! This @AiBlckbx demo shows the advantages of coding significantly better w/ R1 671B & shows our speed against GPUs 😊. In CyberCoder, BlackBox uses R1 to improve the performance of coding agents, a primary use case for #DeepSeekR1 671B. pic.twitter.com/73o7oKKLVz
— SambaNova (@SambaNovaAI) 13 février 2025DeepSeek-R1 671B is on our Cloud! This @blackboxai demo shows the advantages of coding significantly better w/ R1 671B & shows our speed against GPUs . In CyberCoder, BlackBox uses R1 to improve the performance of coding agents, a primary use case for #DeepSeekR1 671B.
-
Developer Deploys App to iOS Device in Part 2
By
–
… and stick around for part 2, where he pushes the app to his iOS device.pic.twitter.com/YthKscLPOT
— Replit ⠕ (@Replit) 13 février 2025… and stick around for part 2, where he pushes the app to his iOS device.
-
Replit enables rapid multi-platform app development for iOS Android Web
By
–
NEW: Replit supports creating apps on your favorite platform.
— Replit ⠕ (@Replit) 13 février 2025
Take your idea and turn it into a iOS, Android, or Web app in minutes. pic.twitter.com/qlOt3JbGoLNEW: Replit supports creating apps on your favorite platform. Take your idea and turn it into a iOS, Android, or Web app in minutes.
-
Building Apps in Minutes with React Native and Expo
By
–
Follow along with @mattppal as he goes from idea to app in under 5 minutes with @expo and @reactnative pic.twitter.com/r7zAptbKRA
— Replit ⠕ (@Replit) 13 février 2025Follow along with @mattyp as he goes from idea to app in under 5 minutes with @expo and @reactnative