All models are bf16. re: context, I know in past we have had restricted usage during spikes, would need to double check for 3.3. We do try to maximise the context length where possible. btw all powered via TGI v3:
COMPUTING
-
Inference API Authentication and Rate Limiting Protection
By
–
all calls to Inference API are authenticated and are rate limited above certain limits to avoid abuse – so no matter the tier you'd be rate-limited if many people use it.
-
Python Tools for Scientists: Anaconda, JupyterLab, and Scientific Libraries Guide
By
–
#Python Tools for Scientists: An Introduction to Using Anaconda, JupyterLab, and Python's Scientific Libraries => Get it at http://
amzn.to/3WrBmZf [700+ page book from @NoStarch Press]
———
#Coding #BigData #DataScience #DataScientist #ComputationalScience #MachineLearning #AI -
MBP M2 Pro Upgrade for Local AI Model Performance
By
–
upgrading from a MBP M2 Pro 24GB – Mostly upgrading because it would overheat quite a lot when playing with local models
-
Modern Computer Vision with PyTorch Deep Learning Second Edition Released
By
–
2nd Edition, 746 pages, massive! Modern #ComputerVision with #PyTorch #DeepLearning — from practical fundamentals to advanced applications and #GenerativeAI: http://
amzn.to/3xAkB7X v/ @PacktPublishing ————
#DataScience #MachineLearning #AI #ML #GenAI #DataScientist -
Secure Smart IoT Using Blockchain and Artificial Intelligence
By
–
"Secure and Smart Internet of Things (#IoT): Using #Blockchain and Artificial Intelligence (#AI)" http://
amzn.to/3gT2Kye
————
#BigData #IIoT #DataScience #MachineLearning #IoTSecurity #DataSecurity #Infosec #CyberSecurity #Edge #EdgeComputing -
Pretraining Vision and Large Language Models in Python on AWS
By
–
Pretrain Vision and #LLMs (Large Language Models) in #Python — Techniques for building & deploying foundation models on #AWS: http://
amzn.to/3pT9seD v/ @PacktPublishing ————
#AI #MachineLearning #DeepLearning #MLOps #BigData #DataScience #NLProc #ComputerVision #DataScientist -
SCHED_FIFO Linux Scheduling vs CUDA Stream Priorities
By
–
I love SCHED_FIFO on Linux not being wish-washy about scheduling — if a higher priority can run, it preempts anything lower Right Now, and it won’t stop until it yields, or something even higher takes it. CUDA stream priorities are more like traditional OS priority hints. Better
-
Integrated Large Models Combined with Good Old Fashioned Engineering
By
–
New computer systems perspective on integrated (modular and hybrid) large models with good old fashioned engineering (GOFE). @UCBerkeley @Berkeley_EECS @Databricks https://
arxiv.org/abs/2412.05299