Seems great, right? Public data is plentiful online, we can just download tons of it, pretrain our models with this public data, and do fine-tuning privately! Privacy is solved! Of course not, and we highlight three (orthogonal) considerations for these settings. 5/n
DATA
-
Pre-training ImageNet Fine-tuning CIFAR-10 Privacy
By
–
The good news is, public data can help dramatically! Pre-training on ImageNet (without privacy constraints) and fine-tuning on CIFAR-10 (privately) gets us up to 95%+ (figure from the same paper). Takes it from bad (and in some cases, unusable) to pretty good! 3/n
-
RLHF automates human evaluation and generates tuning data
By
–
Later, reinforcement learning was used to automate the task of human evaluation as well. This yielded a endless fountain of tuning data that could be produced entirely by machine — a process known as RLHF.
-
Tuning models on filtered self-generations for smarter AI
By
–
By tuning the model on its own generations, filtered to those deemed perfect by human evaluators, greater volumes of data could be used, yielding a more intelligent and obedient model.
-
Random Initialization Achieving 94% Accuracy in Classification
By
–
If you can find a random initialization that by coincidence happens to get >94% of 60,000 10-class classification decisions correct then yes, that counts.
-
Instruction tuning merges human will into AI controlled by awkward kid
By
–
Instruction tuning / RLHF is technically a Human Instrumentality Project, merging the preferences of countless humans to form an oversized, living amalgam of our will. We then hand control of it to a random, socially awkward kid and hope for the best.
-
Five Cybersecurity Risk Pillars for All Organization Sizes
By
–
New! #Security For Any Size My #POV – 5 #CyberSecurity Pillars of #Risk https://
bit.ly/SecurityForAll @KingstonTechBiz #Security #IoT #5G #100DaysOfCode @Shi4Tech @Paula_Piccard @ipfconline1 @rafibloom73 #AI @elenacarstoiu #innovation @1DavidClarke #data @michaeldacosta #SME -
Citizen Data Scientists: Why Professional Expertise Matters Most
By
–
Would you let a “citizen surgeon” operate on you? As data science evangelist Kjell Carlsson says, important data science initiatives require professionals, not #citizendatascientists. Learn more in @alex_woodie
’s piece for @Datanami: https://
domino.buzz/3h9rjtt -
New Math for Machine Learning and Data Science Course Launched
By
–
New course announcement! Math for Machine Learning and Data Science. Designed to make learning AI's math easy: ∙ Linear algebra ∙ Probability & Stats
∙ Calculus Will help learners build better ML systems and pass job interviews! -
Steel manufacturer uses AI to reduce scrap rates and forecast supply
By
–
Steel manufacturer, @NIM_Group, turned to #AI to reduce scrap rates and forecast supply more accurately, saving costs and preventing excess inventory and lost sales. Learn more https://
bit.ly/3UYUDAP