ModernBERT ftw! @answerdotai & @LightOnIO killing it!! > ModernBERT-base: 22 layers, 149M params
> ModernBERT-large: 28 layers, 395M params > 2 trillion tokens of English and code data.
> Up to 8,192 tokens, ideal for processing long documents
> RoPE for long-context support
@reach_vb
-
ModernBERT: New Open-Source Language Model from Answer.ai
By
–
-
New Hardware Gains Enable Faster LLM and VLM Deployment
By
–
This is going to be so, so fun to plug LLMs & VLMs with
— Vaibhav (VB) Srivastav (@reach_vb) 17 décembre 2024
> 67 INT8 TOPS (1.7x increase)
> 102GB/s memory bandwidth (2x increase)
pic.twitter.com/ByRYOjdLcoThis is going to be so, so fun to plug LLMs & VLMs with > 67 INT8 TOPS (1.7x increase)
> 102GB/s memory bandwidth (2x increase) -
Mosaic ML Achieves Major Milestone Recognition
By
–
Massive congrats to Mosaic ML team! – you deserve this and more!
-
Falcon 3 Language Models Released: 1B to 10B Parameters
By
–
Falcon 3 is out! 1B, 3B, 7B, 10B (Base + Instruct) & 7B Mamba, trained on 14 Trillion tokens and apache 2.0 licensed! > 1B-Base surpasses SmolLM2-1.7B and matches gemma-2-2b
> 3B-Base outperforms larger models like Llama-3.1-8B and Minitron-4B-Base
> 7B-Base is on par with -
Microsoft Releases Markdown Processing Tool on GitHub
By
–
Check it out here: https://
github.com/microsoft/mark
itdown
… -
Microsoft Open Sources MarkItDown File to Markdown Converter
By
–
Microsoft open sourced MarkItDown – convert files to Markdown – perfect for using with LLMs!
-
Alibaba Qwen’s Significant Impact on Open Source AI Development
By
–
The impact @Alibaba_Qwen has had to open AI/ ML is understated! Literally paving way to SoTA! 🤯 https://t.co/VJlpZ5QvHd pic.twitter.com/kc9V6oG9Zd
— Vaibhav (VB) Srivastav (@reach_vb) 16 décembre 2024The impact @Alibaba_Qwen has had to open AI/ ML is understated! Literally paving way to SoTA!