Is the trick of initializing with real tokens the secret to making all prefix tuning approaches competitive? BTW @EyubogluSabri I don't see DoRA mentioned in your paper – I wonder if that would close the gap a bit?
MACHINE LEARNING
-
LoRA vs DoRA vs Prefix Tuning: Fine-tuning Method Effectiveness
By
–
The big open question for me is why LoRA (and particularly DoRA) has been the most successful for fine-tuning, but it worked so badly for this use case. Or maybe the opposite – why did prefix tuning work so well?
-
Databricks Summit: Hands-On AI Agent and Data Labs
By
–
At #DataAISummit this week, attendees took part in hands-on labs across a range of technical sessions—gaining practical experience and leveling up their skills on the Databricks platform. Here’s a quick look at what they learned: AI Agent Workflows AI-Powered Data
-
Fast JAX Implementations Delivering State-of-the-Art TPU Performance
By
–
These implementations are very fast — when run with JAX, you'll get state of the art performance. And they all run on TPUs.
-
KerasHub: 70+ Pretrained Models Supporting JAX, TF, PyTorch
By
–
KerasHub is a collection of over 70 popular pretrained model architectures — LLMs, VLMs, image generation models, etc — that work with JAX, TF, PyTorch. They all support HuggingFace checkpoints — you can load any HF model with them for the corresponding architecture.
-
Hundreds of AI Model Checkpoints from Scaling Research
By
–
I now realized I probably have dozens of models there (maybe a hundred?) due to a dump of many checkpoints from one scaling paper, plus ul2 and flan-t5/ule series. And also umt5.
-
12 Essential Machine Learning Algorithms You Should Know
By
–
Want to build real ML solutions? Here are 12 essential #MachineLearning algorithms you should know: 1. Naive Bayes
2. SVM
3. Decision Trees
4. K-Means
5. Linear/Logistic Regression
6. PCA
7. Random Forest
8. Q Learning
9. Dimensionality Reduction
…and more! Save this -
Unity Catalog Metrics and Discover Transform Business Data Management
By
–
Today, at #DataAISummit @matei_zaharia introduced major innovations that expand Unity Catalog to business users!
– Unity Catalog Metrics: A single source of truth for business metrics across data engineering, ML and BI
– Unity Catalog Discover: New curated internal marketplace -
DeepMind AI Weather Lab: Tropical Cyclone Prediction Preview
By
–
You can play with the preview version at https://
deepmind.google.com/science/weathe
r?zoom=2.5&lat=0&lng=60&showTracks=true&showEnsembleTracks=false&showEnsembleMarkers=false&showCyclogenesisMarkers=false&showCones=false
… More details in the blog post at: https://
deepmind.google/discover/blog/
weather-lab-cyclone-predictions-with-ai/
… Research paper: https://
storage.googleapis.com/deepmind-media
/DeepMind.com/Blog/how-we-re-supporting-better-tropical-cyclone-prediction-with-ai/skillful-joint-probabilistic-weather-forecasting-from-marginals.pdf
…