AI Dynamics

Global AI News Aggregator

About

Deduplicating Redundant AI-Generated Output Data

FUTURE WORK – deduplication even though i varied the random seed and used temperature, a lot of the outputs are highly redundant it would be prudent to deduplicate, i bet there are only 100k or fewer mostly-unique examples here

→ View original post on X — @jxmnop