the « internet data lottery »: improvements in AI/ML coming from the path of largest web data availability rather than most scientifically sound direction modalities (text) win the lottery when they happen to be the widest available on the web even if sub-optimal for some tasks
@thom_wolf
-
Galactica Model Repetition Issues Decoding Parameters Analysis
By
–
I’m a bit wondering why we see so much repetitions in the outputs of Galactica people share. If it comes from the model itself or the demo decoding parameters
-
Hugging Face Releases 1.3T Parameter Mixture-of-Experts Model
By
–
We’ve heard mixture-of-experts (MoE) were in the air (GPT4??) so we’ve just added the first one in the transformers library for you to play with 🙂 With for nothing less than a 1.3 trillion parameters checkpoint model on the hub! The largest model on the hub at the moment
-
Self-examination in AI science models for error detection
By
–
what would be cool with science models like Galactica would be to add self examination of the inputs like the coming Codex model, where the model can inspect it’s theorem proofs or essays and spot errors
-
Hugging Face as ML Discovery and Sharing Platform
By
–
Yes that’s exactly the plan! I think HF could be an amazing platform for ML discovery, sharing and discussion. We see it already taking off cc @julien_c @ClementDelangue
-
Twitter ML Noise Forces Shift to Full Paper Reading
By
–
With twitter ML becoming harder and harder to grasp in the noise, I feel slightly blind. It’s crazy how quickly we become dependent on some of our tools. I’ve switched to reading in full papers instead of skimming through discussions about them. Maybe it’s actually for the better
-
Killing Low-Impact Projects: Strategic Portfolio Management
By
–
Same as not killing projects that don’t have any real impact, very hard to do but if you don’t do it along the way, you’re setting you up for tough times
-
Over-hiring risks during economic growth periods
By
–
Of course the second. Over-hiring when everything is going well is so easy a temptation
-
Tech Icons Crisis: Elon’s Leadership Under Fire at Twitter
By
–
however twitter will end –and I hope it survives– Elon doesn’t look like an amazing team leader in this mess Elon, Zuck, SBF…. it’s the week of falling tech icons
-
Team structure and ethical considerations in AI dataset creation
By
–
I would say: smaller teams of full time people in a single org, simpler performance goals, more paid compute, no specific goals on ethical sourcing or social impact of the dataset creation process