Congratulations on the stellar release! The model checkpoints and a detailed report – truly Christmas is here!
CODE
-
Space Engineers 2 and AI People Development Progress Update
By
–
I just published my new weekly dev diary where I share my work on Space Engineers 2 and AI People over the past few weeks: https://
blog.marekrosa.org/2024/12/mareks
-dev-diary-2024-12-26.html
… @SpaceEngineersG -
Reusable Components: Key to Creative Innovation Success
By
–
Even Walt Disney will tell you that designing reusable components is the key to success!
— Pascal Bornet (@pascal_bornet) 26 décembre 2024
By embracing our creative capacities, we can build a better future. Learn more, join the IRREPLACEABLE community, and read the Book: https://t.co/XanuKiNoY6
Thanks Ralph!#inspiration… pic.twitter.com/NLaOxBkwoyEven Walt Disney will tell you that designing reusable components is the key to success! By embracing our creative capacities, we can build a better future. Learn more, join the IRREPLACEABLE community, and read the Book: https://
zurl.co/Z47sz Thanks Ralph!
#inspiration -
Model Configuration and Next Predict Layers Parameter Discussion
By
–
yeah, config.json + modeling looks pretty much the same. Still no reference of “num_nextn_predict_layers” in modeling
-
AI Hackathons: Google Cloud MLB and Windows Snapdragon Opportunities
By
–
#AI Hackathons hosted by @devpost … Google Cloud x MLB(TM) Hackathon – Building with Gemini Models PRIZES: $98,700 DEADLINE: Feb 4, 2025 Build the future of baseball using Google Cloud.
JOIN THE HACKATHON:
https://
bit.ly/googlecloudaic
hallengei
… Windows on Snapdragon AI Hackathon -
Comprehensive Data Scientist Handbook: Essential Learning Resources
By
–
GitHub – andresvourakis/data-scientist-handbook: This is a repo with links to everything you'd ever want to learn about data science https://
bit.ly/3N6nZvo
#AI #MachineLearning #DeepLearning #LLMs #DataScience -
MoE Gate Function Switch From Softmax To Sigmoid
By
–
Thanks for the correction, my logic for comparing defaults in config was a bit faulty In general, the thing to note here is that they switched the MoE gate function to Sigmoid (instead of Softmax) – interestingly they tried this in DeepSeekVL earlier. In addition they have a
-
Model Checkpoint Configuration Values Analysis
By
–
digging into config.json of both the model checkpoint here, these are actual values picked up by the model vs, defaults above:
-
Comparing MoE Architecture: v3 vs v2.5 Model Configurations
By
–
Looking at the config.json for both the models:
v3 (left) vs v2.5 (right) Interesting things: MoE related:
v3: "moe_intermediate_size": 2048, "n_routed_experts": 256, "n_shared_experts": 1, "num_experts_per_tok": 8 v2: "moe_intermediate_size": 1536, "n_routed_experts": 160,