– And good news! More compute = more confidence as well. (However, more compute made one model become more confident in wrong answers too. Something to keep an eye to make sure model confidence doesn’t tip into overconfidence.)
RESEARCH
-
Model Confidence Critical for High-Stakes AI Applications
By
–
– Higher compute and confidence were extra important for model performance when wrong answers had serious consequences (like when they were weighted 20x). You wouldn't want a 51%-sure diagnosis of a terminal illness.
-
Teaching Models Confidence and Uncertainty Recognition
By
–
We know more compute results in higher accuracy, but are the models more confident those answers ARE accurate too? And how do we teach them when to say “I don’t know”? That’s what the research team wanted to find out.
-
Compute Budget and Confidence Thresholds Impact Model Math Performance
By
–
In the study, they measured how different combinations of compute budget and confidence thresholds (being at least 50% sure of the answer, etc.) affected models’ performance on a benchmark math test.
-
Risk Penalty Weights Impact on Model Performance
By
–
They also experimented with different risk levels. On one end of the spectrum: no penalty for wrong answers. On the other: the penalty of wrong answers weighted 20x more than the reward for correct ones. What they found:
-

Johns Hopkins: LLMs Must Know When to Abstain
By
–
You can't just be right, you have to know you're right. Good advice for LLMs, according to new Johns Hopkins research. Sometimes no answer is better than a wrong one – life or death choices in medicine, for example, or big financial decisions.
-
Inquiry on DeepSearch and WebSearch Release ETA and Comparison to Gemini
By
–
What is the ETA for the DeepSearch and WebSearch release? Will it be similar to Deep Research on Gemini?
-
Google Patent Process Takes Years to Complete
By
–
The patent process is hilariously slow . Just found out that my first patent with Google was just approved. The process was so slow that in this time frame I wrote the paper for it, got it accepted, moved countries, got married, left google, had a kid, did a startup, left the
-

SambaNova Cloud Lightning: DeepSeek R1, QwQ, Llama 3.3 Updates
By
–
SambaNova Cloud Lightning Digest, your quick read on the latest dev updates in the SambaNova Community! Fast Deep Research DeepSeek R1 671B with 16k context length QwQ 32B over 350 tps 128K Context Length for Llama 3.3 Feb Community Contest Winners More in
-
Deep Research Improvements Now Available in ChatLLM
By
–
Our deep research is continuously iterating and much more comprehensive and detailed.
— Abacus.AI (@abacusai) 19 mars 2025
Try it out with ChatLLM. pic.twitter.com/uEu2HprN29Our deep research is continuously iterating and much more comprehensive and detailed. Try it out with ChatLLM.