6/ Detecting Pretraining Data from LLMs – explores the problem of pretraining data detection which aims to determine if a black box model was trained on a given text.
LLMS
-
Spectron: End-to-End Spoken Language Model Surpasses Existing Systems
By
–
4/ Spectron – a spoken language model trained end-to-end to directly process spectrograms; it can be fine-tuned to generate high-quality accurate spoken language; surpasses existing spoken language models in speaker preservation and semantic coherence.
-
LLMs Struggle With New Knowledge Understanding and Association
By
–
5/ LLMs Meet New Knowledge – presents a benchmark to assess LLMs' abilities in knowledge understanding, differentiation, & association; benchmark results show (unsurprisingly) that an LLM’s performance when introduced to new knowledge is not satisfactory.
-
Fact-Checking Capabilities of Large Language Models Compared
By
–
2/ Fact-checking with LLMs – investigates fact-checking capabilities of LLMs; shows the enhanced prowess of LLMs when equipped with contextual information; GPT4 outperforms GPT-3, but accuracy varies based on query language & claim veracity.
-
Evaluate LLM Performance: Metrics and Benchmarking Strategies
By
–
Master LLMs: Top Strategies to Evaluate LLM Performance In this video, we look into how to evaluate and benchmark Large Language Models (LLMs) effectively. Learn about perplexity, other evaluation metrics, and curated benchmarks to compare LLM performance. With practical tools
-
Robust Prompts Maintain Value Across LLM Variations
By
–
Yes. But some prompts are quite robust to noise (and to differences in LLMs), and that's their value.
-
DoReMi: Optimizing Data Mixtures for Faster Language Model Pretraining
By
–
Check out DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining, and some of the papers it cites. By @sangmichaelxie et al.
-
Knowledge Versus Intelligence: Why LLMs Differ From Human Cognition
By
–
No human brain contains as much knowledge as Wikipedia.
But that doesn't make Wikipedia smart.
An LLM that could correctly answer any question whose answer is in Wkipedia would have more knowledge than any human.
But it still would not be nearly as smart as humans. -
Auto-Regressive LLMs: Dumb Yet Knowledgeable and Useful
By
–
My views on how useful Auto-Regressive LLMs can be has not changed.
I've said numerous times that they are dumb and unreliable, yet knowledgeable and useful.
