we spend a lot of time optimizing decently-sized models too.
For anything that is normal-sized (one node, 8 GPUs), torch.compile + FSDP2 should be largely sufficient to get to really good performance.
We've spent hundreds (or probably thousands) of programmer-years working on
OPEN SOURCE
-
Optimizing AI Models with torch.compile and FSDP2
By
–
-
Handling Large Text Inputs with Longformer Transformers
By
–
How to Handle Large Text Inputs with Longformer and Hugging Face Transformers: Let’s learn how to handle large text inputs in the Large Language Model (LLM). Preparation Ensure you have the Transformers and datasets package from Hugging Face… https://
kdnuggets.com/how-to-handle-
large-text-inputs-with-longformer-and-hugging-face-transformers?utm_source=dlvr.it&utm_medium=twitter
… -
Interoperability Path for Open Table Formats Explored
By
–
Explore the path to interoperability with @michaelarmbrust (the original creator of #DeltaLake) & Ryan Blue (an original creator of #ApacheIceberg). They discuss the evolution of open table formats, how Databricks is solving interoperability & more: https://
dbricks.co/4ejg3my -
Targeting checkpointing improvements planned for machine learning
By
–
it's on our list of things to do (more targeting checkpointing, not FSDP). I wrote up a plan / doc early last year about it.
-
Llama 3.2 Launches with Record-Breaking Inference Speed Performance
By
–
Llama 3.2 is here and we're faster than ever! We've been independently verified as the fastest #AI #Inference on @AIatMeta
's Llama 3.2 1B & 3B with … 2470 tokens/sec on 1B 1566 tokens/sec on 3B … all running at full-precision! Start developing -
YouTube-Whisper: YouTube Video Audio Transcription with Whisper
By
–
Youtube-Whisper
— AK (@_akhaliq) 2 octobre 2024
A simple Gradio app that transcribes YouTube videos by extracting audio and using OpenAI’s Whisper model for transcription. Paste a YouTube link and get the video’s audio transcribed into text. pic.twitter.com/c6aNzjNYCiYoutube-Whisper A simple Gradio app that transcribes YouTube videos by extracting audio and using OpenAI's Whisper model for transcription. Paste a YouTube link and get the video's audio transcribed into text.
-
Daily Papers Project Translates and Summarizes Huggingface Research into Korean
By
–
daily_papers_ko This project aims to automatically translate and summarize Huggingface's daily papers into Korean using ChatGPT.
-
GraphRAG-UI: User-Friendly Interface for RAG Text Indexing
By
–
GraphRAG-UI
— AK (@_akhaliq) 2 octobre 2024
GraphRAG-UI is a user-friendly interface for GraphRAG, a powerful tool that uses the Retrieval-Augmented Generation (RAG) approach to index and query large text data. This project supports the latest version graphrag-0.3.3 and aims to provide a convenient management… pic.twitter.com/davBb07LcBGraphRAG-UI GraphRAG-UI is a user-friendly interface for GraphRAG, a powerful tool that uses the Retrieval-Augmented Generation (RAG) approach to index and query large text data. This project supports the latest version graphrag-0.3.3 and aims to provide a convenient management
-
vLLM 0.6.2 now supports SolarPro
By
–
Super happy that @vllm_project 0.6.2 now supports #SolarPro. Check it out! https://
github.com/vllm-project/v
llm/releases
… -
Transform Hugging Face Spaces into Discord Bots with Gradio
By
–
Gradio Bot
— AK (@_akhaliq) 1 octobre 2024
Turn any Hugging Face Space or Gradio application into a discord.js bot. pic.twitter.com/YDTxrV6qUZGradio Bot Turn any Hugging Face Space or Gradio application into a discord.js bot.