Cerebras is starting the year off with a guest presentation! @simula_research
's Luk Burchard will be sharing research on the Graph500 BFS algorithm that was ported over to the Cerebras Software Language (CSL). Register for this community session here: https://
us02web.zoom.us/webinar/regist
er/WN_s23U5O64T5K9H8h0LhnFJg
…
AI
-
Cerebras CSL Graph500 BFS Algorithm Research Presentation
By
–
-
Training a 10M Parameter GPT Model on Shakespeare in 15 Minutes
By
–
We get a ~10M parameter model trained for about 15 minutes on 1 GPU on all of Shakespeare concatenated into one 1MB file. We then sample infinite fake Shakespeare from our baby GPT. Can you spot which one is real? At only 10M params on 1M characters, from-scratch, I hope so 🙂
-
Building Transformers: Self-Attention, Training, and GPT-3 Comparison
By
–
The second ~1hr builds up the Transformer: multi-headed self-attention, MLP, residual connections, layernorms. Then we train one and compare it to OpenAI's GPT-3 (spoiler: ours is around ~10K – 1M times smaller but the ~same neural net) and ChatGPT (i.e. ours is pretraining only)
-
Transformer Attention Mechanism: Baseline Model and Message Passing Introduction
By
–
First ~1 hour is 1) establishing a baseline (bigram) language model, and 2) introducing the core "attention" mechanism at the heart of the Transformer as a kind of communication / message passing between nodes in a directed graph.
-
ChatGPT Money Making Resources Free for 48 Hours
By
–
Making money using ChatGPT is a once-in-a-lifetime opportunity. We organized ALL of the best resources to use ChatGPT into one page. Free for 48 hrs. To get it:
1. Retweet, Follow, Comment 'me'
2. Subscribe to free newsletter in bio (optional) 🙂 & We'll DM it to you, free. -
Happy Birthday NeRF: Celebrating Deep Learning History
By
–
¡Feliz cumpleaños a NeRF! Una de las tecnologías favoritas de mi canal Cómo mola conocer estas pequeñas piezas de la historia del Deep Learning.
-
Chat-LangChain: Extensible AI Documentation Tool for Websites
By
–
got it – https://
github.com/hwchase17/chat
-langchain
… should be pretty extendable to arbitrary websites (is currently langchain documentation, but you can swap out the URL). going to add more instructions for this today! -
Standard Interfaces for LLM and Embeddings Providers
By
–
pretty easy to swap this out! we have standard interfaces for most llm/embeddings providers
-
For embedding retrieval, hallucinating lies beats ‘I don’t know’
By
–
I love how this is a clear use case for hallucination. If the reason you’re generating an “answer” is merely to produce an embedding suitable for retrieving one, saying “I don’t know” is useless and confidently making up lies is better.
-
HyDE: Generating Hypothetical Answers for Better Document Retrieval
By
–
Clever paper — HyDE: Hypothetical Document Embeddings Instead of encoding the user's query to retrieve relevant documents, generate a "hypothetical" answer and encode that. Documents with right answers more similar to wrong answers than to questions. https://
arxiv.org/abs/2212.10496