yeah fp16 is a little more efficient atm for the code as I have it right now but then need gradient scaler ;s
CODE
-
Optimizing minGPT: Performance improvements from 495ms to 102ms
By
–
having fun optimizing minGPT today
– base: 495ms
– zero_grad(set_to_none=True): 492
– torch.jit.script gelu: 463
– OMP_PROC_BIND=CLOSE: 453
– torch.backends.cuda.matmul.allow_tf32: 143
– torch.autocast(torch.bfloat16): 121
– FlashAttention: 102
now: more fused kernels more better -
Splitting minGPT into educational and efficient versions
By
–
Context I realized I have to split up minGPT because I can't properly simultaneously satisfy both 1) educational and 2) efficient in one repo. So I'm separately writing 1) the maximally educational minGPT (+video etc.) and 2) a more efficient (still ~clean) version that has teeth
-
Dealing with GPT-3 Limited Context Window Constraints
By
–
how do you deal with the limited context window of GPT3? afaik, ChatGPT a limit of 8092 tokens, whereas GPT3 is lower (3-4k?) any langchain solution to that sort of thing?
-
MIT Sloan AI in Action: Real-World Use Cases
By
–
The @mitsmr
“AI in Action” column series dives deep into successful use cases that can help other organizations accelerate their #AI progress https://
sloanreview.mit.edu/series/ai-in-a
ction/
… #MachineLearning #DataScience #serverless #100DaysofCode #womenwhocode @FmFrancoise @CatherineAdenle @Shi4Tech -
ChatGPT Streamlines After Effects Script Generation for Creators
By
–
Qué maravilla ChatGPT para crear scripts rápidos para las animaciones de After Effects. Oye, que quiero una matriz de números aleatorios de dimensiones NxM… ¡HECHO EN SEGUNDOS!
-
Real-time ML Feature Store: Merge Data Pipelines and Deploy AI Models
By
–
@abacusai
's Real-time ML feature store lets you merge batch and streaming data pipelines, build features in SQL or Python, and create advanced nested and time travel features for your models. You can deploy your AI models at scale with enterprise-class security and governance. -
Gradient Descent: Essential Optimization Algorithm for Machine Learning Models
By
–
Gradient descent – a crucial tool for anyone working in #machinelearning – is an optimization algorithm commonly used to train models and neural networks. IBM Master Inventor @martinrtp
, illustrates its utility: https://
ibm.co/3hH8zSm —-
#IBM #datascience #AI #ML -
Saving the Titanic Using Azure AutoML
By
–
Saving the Titanic Using Azure AutoML!: This article was published as a part of the Data Science Blogathon. Source:
http://
pixabay.com Introduction State-of-the-art machine learning models and artificially intelligent machines are made of complex… https://
analyticsvidhya.com/blog/2022/11/s
aving-the-titanic-using-azure-automl/?utm_source=dlvr.it&utm_medium=twitter
… -
MLOps Maturity Model Infographic and Best Practices
By
–
#Infographic: #MLOps Maturity Model
Via @ingliguori #BigData #Analytics #DataScience #AI #PyTorch #Python #RStats #TensorFlow #JavaScript #ReactJS #CloudComputing #DevCommunity #DataScientist #DigitalTransformation #Cloud #MachineLearning #100DaysofCode #ArtificialIntelligence