Here's how we do it in @spacy_io : https://
explosion.ai/blog/floret-ve
ctors
… We're interested to try this trick in transformer models. We haven't really tried that yet.
OPEN SOURCE
-
spaCy Floret Vectors: New Approach for Transformer Models
By
–
-
Building a Transformer with simplified layers in nanoGPT
By
–
Yep, that's the one! (as @Thom_Wolf linked earlier too). I'd expect it's possible to build a Transformer with that kind of layer alone, would look much more pleasing. Will see if I can prototype in nanoGPT.
-
Simplified Transformer Architecture with Unified Block Design
By
–
TLDR: A much simpler Transformer with a single type of block wired up to a residual pathway in both parallel and in series is possible but to my knowledge has not yet been convincingly demonstrated. Bit more detail @ https://
github.com/karpathy/rando
mfun/blob/master/transformer_unify.ipynb
… -
Open-sourcing assistance offered for project
By
–
Very cool! We can help on the open-sourcing part if needed
-
Open Source AI Projects for Beginners on GitHub
By
–
You can check this out – https://
projectpro.io/article/open-s
ource-ai-projects-for-beginners-github/517
… -
Collaborating on Open Source Projects for Growth
By
–
2. Collaborate on opensource projects Collaborating with others on opensource projects is a great way to learn new skills and give back to the community. It also gives you the opportunity to apply your knowledge to real-world problems and learn from experienced developers.
-
Lucidrains launches open musiclm pytorch implementation
By
–
That’s correct. I did jump the gun whilst reading the paper and tweeting. Thanks for correcting me. Btw lucidrains is working on an open implementation: https://
github.com/lucidrains/mus
iclm-pytorch
… (just an empty repo for now) -
Top 10 AI Tools and Frameworks You Need to Know
By
–
#AI https://
analyticsinsight.net/top-10-ai-tool
s-and-frameworks-you-need-to-know-in-2023/
… -
Work Simulator Hard Mode Should Be Open Sourced
By
–
Your work simulator is a wild one, super hard mode. You should open source it.
-
Hugging Face Datasets Converter Transfer Kaggle Zenodo
By
–
This wouldn't have been possible without @_nateraw
's brilliant Hugging Face Datasets Converter Use it to transfer any dataset from Kaggle or Zenodo over to the Hub!