DBRX, Mixtral 8x22B, Llama-3 all released within weeks. The open-source AI scene shows no sign of slowing down.
@hardmaru
-
Oracle invests $8bn in Japanese data centers for data sovereignty
By
–
Oracle to spend over $8bn to build more data centers in Japan, hopes to ride the ‘data sovereignty’ wave.
-
RNNs Making a Comeback: State-Space Models Analysis
By
–
RNNs are not dead yet In fact, they are coming back with a vengeance recently. Very nice paper about “The Illusion of State in State-Space Models” (
https://
arxiv.org/abs/2404.08819) and thread -
Model Fine-Tuning for Human Preferences
By
–
Your model, after some fine-tuning for human preferences.pic.twitter.com/lzq5WAqP1b
— hardmaru (@hardmaru) 15 avril 2024Your model, after some fine-tuning for human preferences.
-
OpenAI Expansion to Tokyo: AI Hub Growth in Japan
By
–
Great to see more companies like OpenAI also expanding to Japan. Tokyo is going to become a city with a significant presence in AI. Tokyo has always attracted the ‘right’ kind of people: creative, curious, weird and eccentric. Official announcement here:
-
NeurIPS Introduces Preschool Track for Machine Learning Research
By
–
Announcing NeurIPS Preschool Track This year, we invite preschoolers to submit machine learning research papers.
-
Collective Intelligence Across Biological Scales and Swarm Behavior
By
–
“Collective intelligence is not only the province of groups of animals, and that an important symmetry exists between the behavioral science of swarms and the competencies of cells and other biological systems at different scales.”
-
H100 GPU Pricing in Japan at $2.6 per Hour
By
–
H100 at ~$2.6/hour in Japan. I can get cheaper options, but we’re talking about only a few thousand bucks of savings at this scale.
-
Avoid Multi-Year Compute Contracts, Focus on Innovation Instead
By
–
As an AI R&D company, the last thing I’d do is to lock myself up in multi-year expensive compute contracts. Those crazy fixed-costs will ultimately destroy many so-called “GenAI” startups. Focus on innovation instead. My comment on @steph_palazzolo piece: https://
theinformation.com/articles/why-t
wo-models-are-better-than-one?commentId=22938
… -
Sakana AI’s cost-efficient model merging with limited GPU resources
By
–
“At the time, @SakanaAILabs only had 16 GPUs, which cost $30K per month. That’s pennies compared to the $100M+ it took to train advanced models like GPT-4… Model merging isn’t perfect though. @SakanaAILabs is one of only a few firms who have attempted to automate this process.”