MACHINE LEARNING

→ View original post on X — @gp_pulipaka

Transformers with RNNs: Advanced Machine Learning Techniques

By

@gp_pulipaka

–

18 March 2023 20h42

Transformers with RNNs! #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #NLProc #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Programming #Coding #100DaysofCode https://
geni.us/TF-RNNs

→ View original post on X — @gp_pulipaka

Analytics and Probability in Big Data and Machine Learning

By

@gp_pulipaka

–

18 March 2023 20h42

Analytics & Probability! #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Books #Programming #Coding #100DaysofCode https://
geni.us/Analytics-N-Pr
ob
…

→ View original post on X — @karpathy

Fine-tuning Miscalibration: Model Confidence Doesn’t Match Accuracy

By

@karpathy

–

18 March 2023 18h43

If not careful, fine-tuning collapses entropy relatively arbitrarily, creates miscalibrations, e.g. see Figure 8 from GPT-4 report on MMLU. i.e., if a model gives probability 50% to a class, it is not correct 50% of the time; its confidence isn't calibrated.

→ View original post on X — @karpathy

Gradient-based Learning Becoming Less Common in AI

By

@karpathy

–

18 March 2023 18h43

I'm still intuitively adjusting to the new world where gradient-based learning is less common/desirable. But the trend increases my confidence in an earlier prediction in my earlier "33 years from now" blog post https://
karpathy.github.io/2022/03/14/lec
un1989/
…

→ View original post on X — @karpathy

Base LLMs as Strong Few-Shot Classifiers Without Fine-tuning

By

@karpathy

–

18 March 2023 18h43

Base LLMs (non-finetuned) make very strong few-shot classifiers. Describe task in English, give few examples, read off the label probabilities on test example. No gradient-based optimization necessary. It brings a cannon to a knife fight but is fast, convenient, strong baseline.

Parameters Storage: Understanding Models Beyond Traditional Databases

By

–

18 March 2023 17h53

Only the parameters. The data isn't stored (though that distinction isn't that important). In that sense it's more like a regression curve than a dict. It is not an exact superset of a RDBMS.

KerasNLP Reaches 30,000 Monthly Downloads Milestone

By

–

18 March 2023 17h27

KerasNLP adoption is still early days, but doing ok! We just crossed 30,000 downloads / month. https://
keras.io/api/keras_nlp/

Neural Networks as Interpolative Databases with Natural Language Interfaces

By

–

18 March 2023 17h01

You can retrieve not just what was seen at training time, but arbitrary combinations of it. It's an interpolative database and program store, with a natural language interface.

LLMs as Queryable Databases, Not Mere Autocomplete Systems

By

–

18 March 2023 16h58

"It's autocomplete" is not a helpful analogy to understand LLMs. A LLM is more like a database that lets query information in natural language. You can query both knowledge, and "patterns" (associative programs seen in the training data, that can be applied to new inputs).