Super proud to announce my new book: the LLM Engineer's Handbook I think we've built something special with @iusztinpaul and Alex Vesa, focused on best engineering practices, reproducible pipelines, and end-to-end deployment. Basically everything that is currently lacking in
EDUCATION
-
AI21 Labs Offers Course on Hybrid SSM-Transformer Architecture
By
–
Congrats Andrej! We'd love to help you create a course on building a hybrid SSM-Transformer architecture and all of the awesome benefits of that.
-
Free Educational Content with Revenue from Services
By
–
Good question I do want Eureka Labs to be a proper, self-sustaining business but I also really don't want to gatekeep educational content. My default thinking is that the content itself is free and permissively licensed, the revenue comes from everything else, e.g. running the
-
Andrej Karpathy Launches Eureka Labs, AI Education Startup
By
–
Andrej Karpathy, a founding member of openai who left the company earlier this year, is starting an AI and education company called Eureka Labs. Will be interesting to see where this goes.
-
AI’s Impact on Education and Teaching Best Practices
By
–
Congrats! AI is going to have a huge impact on education & I know it's something you're passionate about (seeing how you organized cs231n years ago when I gave a guest lecture it was clear you put lots of thought into the best way to teach people different kinds of material)
-
Eureka Labs: AI-Native School with LLM101n Course
By
–
⚡️ Excited to share that I am starting an AI+Education company called Eureka Labs. The announcement: — We are Eureka Labs and we are building a new kind of school that is AI native. How can we approach an ideal experience for learning something new? For example, in the case of physics one could imagine working through very high quality course materials together with Feynman, who is there to guide you every step of the way. Unfortunately, subject matter experts who are deeply passionate, great at teaching, infinitely patient and fluent in all of the world's languages are also very scarce and cannot personally tutor all 8 billion of us on demand. However, with recent progress in generative AI, this learning experience feels tractable. The teacher still designs the course materials, but they are supported, leveraged and scaled with an AI Teaching Assistant who is optimized to help guide the students through them. This Teacher + AI symbiosis could run an entire curriculum of courses on a common platform. If we are successful, it will be easy for anyone to learn anything, expanding education in both reach (a large number of people learning something) and extent (any one person learning a large amount of subjects, beyond what may be possible today unassisted). Our first product will be the world's obviously best AI course, LLM101n. This is an undergraduate-level class that guides the student through training their own AI, very similar to a smaller version of the AI Teaching Assistant itself. The course materials will be available online, but we also plan to run both digital and physical cohorts of people going through it together. Today, we are heads down building LLM101n, but we look forward to a future where AI is a key technology for increasing human potential. What would you like to learn? — @EurekaLabsAI is the culmination of my passion in both AI and education over ~2 decades. My interest in education took me from YouTube tutorials on Rubik's cubes to starting CS231n at Stanford, to my more recent Zero-to-Hero AI series. While my work in AI took me from academic research at Stanford to real-world products at Tesla and AGI research at OpenAI. All of my work combining the two so far has only been part-time, as side quests to my "real job", so I am quite excited to dive in and build something great, professionally and full-time. It's still early days but I wanted to announce the company so that I can build publicly instead of keeping a secret that isn't. Outbound links with a bit more info in the reply!
→ View original post on X — @eurekalabsai, 2024-07-16 17:25 UTC
-
ALT 2025 Conference in Milan: Submit Learning Theory Research
By
–
ALT 2025 is going to be in Milan, Italy , in February 2025. Po-Ling Loh and I are the PC co-chairs. Submit your best learning theory work!
-
Backward Pass: Computing Output Layer Gradients and Weight Updates
By
–
Backward Pass: First compute the gradients of the output layer. Loss = (y – output) Gradient of Loss = (y – output) * sigmoid_derivative(output) Now calculate d_W2 which is gradient of the loss function with respect to W2. d_W2 = hidden_output.T • Gradient of Loss
-
Calculate Gradients dW1, db2, db1 for Neural Network Backpropagation
By
–
Similarly calculate d_W1, d_b2 & d_b1 dW1: Gradient of the loss function wrt W1 d_b2: Gradient of the loss function wrt b2(bias of neuron in output layer) d_b1: Gradient of the loss function wrt b1(bias of neuron in hidden layer)
-
Forward Pass in Neural Networks: Hidden Layer Calculation and Activation
By
–
Forward Pass: Here the input data is passed through the neural network to obtain the predicted output. In forward pass, First calculate the output of the hidden layer. hidden_output = X•W1 + b1 Then apply the sigmoid activation to the output. output = sigmoid( (X•W1) + b1)