Isn't it also related to diversity in your dataset? 2B = not diverse enough to scale, 10B = maybe okay?
GENERATIVE AI
-
Tencent Releases Hunyuan Large Language Model with Multimodal Capabilities
By
–
Paper: https://
arxiv.org/abs/2411.02265 Model: https://
huggingface.co/tencent/Tencen
t-Hunyuan-Large
… -
Hunyuan-Large Post-Training Pipeline: SFT and DPO Strategy
By
–
Interestingly, Hunyuan-Large follows a traditional post-training pipeline with SFT and DPO. SFT data is sourced from public sources and evolved to increase complexity Data quality is ensured through 3 stages: rules → 70B critique model → human review If I understood
-
Trump’s Victory Boosts e/acc Movement and AI Deregulation
By
–
Without wishing to make a political classification, it is quite clear that Donald Trump has given e/acc a major boost. This means less regulation and freer development for AI. Trump is backing a deregulated approach and has announced that he will repeal Biden's AI regulation
-
Qwen Models: Why Practitioners and AI Teams Follow Their Releases
By
–
Two type of teams carefully follow Qwen releases:
– practitioners cutting through the hype to find the best models to run locally
– teams training state-of-the-art LLMs who knows how hard it is to beat Qwen models (notice how they are often absent from comparisons…) I’m at -
Hunyuan-Large: 389B Parameter Open-Source MoE Model
By
–
introduc ing Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens.
-
Tencent Releases Hunyuan-Large Open-Source MoE Model
By
–
Tencent released Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters https://
llm.hunyuan.tencent.com https://
github.com/Tencent/Hunyua
n-Large
… https://
huggingface.co/tencent/Hunyua
n-Large/tree/main
… https://
huggingface.co/spaces/tencent
/Hunyuan-Large
… https://
arxiv.org/abs/2411.02265 -
Minor issues dominate AI narrative discourse globally
By
–
Minor. This is the noisiest and most fought over narrative space in the world.
-
Create AI Chatbot Apps Instantly with SambaNovaAI and Gradio
By
–
Developers can now make ai chatbot apps with one button click
— AK (@_akhaliq) 5 novembre 2024
using @SambaNovaAI cloud and @Gradio integration with @huggingface Spaces
example with @AIatMeta llama 3.1 8B instruct pic.twitter.com/hyuxnOpd3yDevelopers can now make ai chatbot apps with one button click using @SambaNovaAI cloud and @Gradio integration with @huggingface Spaces example with @AIatMeta llama 3.1 8B instruct
-
GPT-4o Advanced Voice Mode Launch Announcement
By
–
Open GPT-4o Advanced Voice Mode ⚡️ https://t.co/Yae2nFv3gF
— Vaibhav (VB) Srivastav (@reach_vb) 5 novembre 2024Open GPT-4o Advanced Voice Mode