AI - AI Dynamics

→ View original post on X — @soumithchintala

AI Research Integrity Crisis: Academic Fraud Parallels Emerge

By

@soumithchintala

–

26 June 2023 16h13

What a hustle!
I feel sorry for the students on the paper for starting their journey in such a careless, low-integrity environment. The last time AI hit a bubble was ~2014, and this paper reminds me of Baidu cheating on the Imagenet competition. Similar playbook — threw a https://
x.com/ML_PhDer/statu
/ML_PhDer/status/1672750801234857984
…

→ View original post on X — @marktabnet

HuggingChat Python API: Free Open Source ChatGPT Alternative

By

@marktabnet

–

26 June 2023 16h07

HuggingChat is a free and open source alternative to commercial chat offerings such as ChatGPT. The unofficial Python API gives you immediate access, without signup, for free. HuggingChat Python API: Your No-Cost Alternative – KDnuggets https://
bit.ly/3NgE3u7

→ View original post on X — @fogoros

Condition-Based Maintenance Strategy with Real-Time Monitoring

By

@fogoros

–

26 June 2023 16h03

Learn how to implement a needs-driven filter replacement strategy and shift from time-based to condition-based maintenance leveraging @ifm_USA’s #moneo Real-Time Monitoring System. https://
ow.ly/WRn950OOlTT #sponsored #ifm_iiot #digitaltransformation #iiot #industry40 #automation

Collaborative LLM Project with Falcon and LLaMA Teams

By

–

26 June 2023 15h50

This was a really cool project with @clefourrier @nathanhabib1011 @osanseviero and a lot of help from @slippylolo from the Falcon team and the amazing LLaMA team, in particular Xavier Martinet, Aurélien Rodriguez and Sharan Narang!

MMLU Benchmark Updated in Open LLM Leaderboard

By

–

26 June 2023 15h46

25/ THE END PS: Please, come comment/discuss at https://
huggingface.co/spaces/Hugging
FaceH4/open_llm_leaderboard/discussions/82
… PPS: During the writing of this post, @Ollmer & the community have done an amazing work updating MMLU in the Harness to make it similar to the original. We're thus now updating the leaderboard based on it!

Open Standardized Benchmarks Essential for AI Model Evaluation

By

–

26 June 2023 15h44

24/ That's why open, standardized, reproducible benchmarks such as the EleutherAI Harness https://
github.com/EleutherAI/lm-
evaluation-harness/
… or Stanford HELM https://
github.com/stanford-crfm/
helm/
… are invaluable to the community. Without them comparing results across models/papers would be impossible, stifling research!

MMLU Scores Incomparable: Evaluation Implementation Details Matter

By

–

26 June 2023 15h44

23/ …not at all comparable even if they're both called MMLU & evaluated on same dataset Takeaway? Evaluations are strongly tied to implementations–down to minute details. A mere indication of "MMLU score" gives almost no information about how you can compare these numbers

LLaMA 65B Evaluation Discrepancies Between Implementations

By

–

26 June 2023 15h43

22/ Say you've trained a perfect LLaMA 65B reproduction & evaluated it with EAI harness (score 0.488). Comparing it to the published number (evaluated w. original implementation, score 0.637), it's a 30% difference so you're likely thinking "Oh no " But these numbers are…

Model Evaluation Methods Significantly Impact Scores and Rankings

By

–

26 June 2023 15h43

21/ Now let’s compare the model scores on these three possible ways to evaluate the models We can see that for the same dataset, both absolute scores and model rankings are very sensitive to the evaluation method we decide to use.