Yeah I agree. When you replicate entire blocks of several layers, the input/output difference dramatically changes, which might degrade the quality of the final output. Replicating individual layers in a distributed way tries to address this issue.
CODE
-
Echo Merge: Optimizing Layer Replication in AI Models
By
–
For the echo merge, the idea is that self-merges might be slightly "insane" because replicating blocks of several layers disrupts the information flow. By only replicating individual layers in a distributed way, I'm trying to minimize the input/output difference they introduce.
-
Echo Merge Experiment Tests Self-Merge Information Flow
By
–
I don't expect it to have superior performance in general, but in some narrow domains. The echo merge is an experiment to check if the "insanity" of these self-merges comes from a broken information flow.
-
CUDA Mode Rebrands to GPU Mode Community
By
–
"CUDA Mode" has grown up! 😀 It's now "GPU Mode". https://
discord.gg/gpumode -
Turbo LoRA: Fast Inference with Customized Models
By
–
Last chance to save your spot Learn how to get the best of both worlds: super fast #inference + high quality models customized for your use case! Join us tomorrow, to learn about Turbo LoRA, a new approach to fine-tuning that increases model #throughput by 2-3x, lowers
-
alphaXiv Becomes arXiv Labs Project with Chrome Extension
By
–
Hi Alessandro! We're in the process of becoming an arXiv Labs project, upon which arXiv will include linking to alphaXiv! Until then, we have a Chrome extension with automatic linking when reading papers on arXiv. Feel free to check it out here:
-
SambaNova Cloud Offers Free Access to Meta’s Llama 3.1 405B
By
–
Thank you, @rohanpaul_ai for being a great resource for the #dev community In case you missed it, our recently launched SambaNova Cloud offers developers the chance to use @AIatMeta
's Llama 3.1 405B model for free! Start developing http://
cloud.sambanova.ai -
ChatGPT o1-preview enables coding for non-experts
By
–
ChatGPT o1-preview is crazy.
— God of Prompt (@godofprompt) 23 septembre 2024
Just imagine the things you can code with 0 knowledge.
Many possibilities are ahead of us! pic.twitter.com/nQlHQxCwaJChatGPT o1-preview is crazy. Just imagine the things you can code with 0 knowledge. Many possibilities are ahead of us!
-
ChatGPT generated a complete ping-pong game from a prompt
By
–
1/ All I did was prompt it the following: "Create a ping pong game for me." ChatGPT quickly went ahead and coded the ping pong game in seconds.
-
Ping pong game created with ChatGPT o1-preview
By
–
I created a ping pong game with ChatGPT o1-preview in one minute.
— God of Prompt (@godofprompt) 23 septembre 2024
Here's a short 🧵: pic.twitter.com/L3hbNUgJ4XI created a ping pong game with ChatGPT o1-preview in one minute. Here's a short :