Thanks! The relative position of normalization is one of the few things that changed about the original transformer architecture. I think it's not exactly clear where it should be placed. Most transformers for texts use post-norm(one above) whereas vision transformers tends to
CODE
-
Document splitting with metadata preservation for markdown
By
–
a lot of people have been asking for a version of this: split documents, but keep metadata about what section each chunk comes from starting with markdown, but pretty easily extendable to other doc types
-
OpenAI Functions Emergency Webinar with Builders
By
–
EMERGENCY OpenAI Functions Webinar @OpenAI functions is the hottest new thing in the LLM world. Excited to do a last-minute webinar on it Featuring Atty Eleti from OpenAI, and then two builders: @fpingham and @jxnlco Register
-
Understanding DeepMind’s Sorting Algorithm
By
–
Understanding DeepMind's Sorting Algorithm https://
bit.ly/3NALIEU #AI #MachineLearning #DeepLearning #LLMs #DataScience -
AI-Powered Coding: New Cohort Starting September
By
–
It's truly incredible what people can make when they use AI to help them code. If you're interested in building something like this, we're doing a new cohort in September! Sign up here: https://
chatbot-course.com -
80 Students Built AI Chatbots in 30-Day Cohort Course
By
–
In May I ran my first cohort-based course: How to Build an AI Chatbot. 80 students built their own AI chatbot in 30 days—using ChatGPT to help them code. I was blown away by some of the projects students produced. Here are a few of them:
-
Compute Optimal Fine-Tuning Experiments: Peer-Reviewed Guides
By
–
Any guide/ resources for designing compute optimal experiments? Modality doesn't matter, essentially looking for a peer-reviewed way of running compute optimal, knockout style fine-tuning experiments.
-
VideoComposer: Controllable Motion Video Synthesis Models Released
By
–
VideoComposer: Compositional Video Synthesis
— AK (@_akhaliq) 17 juin 2023
with Motion Controllability models are out!
paper page: https://t.co/jb0kyDGCVH
github: https://t.co/GARo57U90t
The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in… pic.twitter.com/7RFfNJq3TVComposer: Compositional Synthesis
with Motion Controllability models are out! paper page: https://
huggingface.co/papers/2306.02
018
…
github: https://
github.com/damo-vilab/vid
eocomposer
… The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in -
Democratizing AI: Empowering Everyone to Work With Their Own Data
By
–
Everyone-the mega corporations, a mom & pop store, a student running biology experiments-now has their own data. I’d like everyone empowered to have AI work on their own data. Coding (even a little) lets people do this much more flexibly, which is why I advocate teaching that.
-
BERT and RoBERTa remain top models for classification tasks
By
–
BERT/RoBERTa are still among the best models for classification tasks. Even Twitter uses them here for their platform.