The astute reader will notice I never mentioned computation. Indeed, as stated, it's not clear how to compute this estimator efficiently (or at all!). That's where most of the technical work comes in. I'll leave you with this theorem, check out the rest of the paper. 10/n
MACHINE LEARNING
-
Private Robust Gaussian Estimation Framework Near-Optimal Sample Complexity
By
–
So what can we do with this framework? The most interesting application is estimating Gaussians privately AND robustly: the resulting algorithm nails it, with a near-optimal sample complexity. Other potential applications include mean estimation, regression, etc. 9/n
-
Robust Estimators and Exponential Mechanism for Parameter Scoring
By
–
The notation is a dense, but the algorithm is simple. The score of a parameter vector θ (wrt a dataset) is the minimum number of datapoints that must be changed to make the robust estimator output (approximately) θ. Feed this score into the exponential mechanism. 8/n
-
Robust Algorithms Enable Private Algorithm Design
By
–
In our work, the main conceptual result shows the other direction: a robust algorithm can be used to design a private algorithm. 7/n
-
Differential Privacy Frameworks for Robust Statistical Estimators
By
–
Most have been instance-specific, e.g., here's a particular robust estimator, let's privatize it. Some have proposed broader frameworks, e.g., https://
arxiv.org/abs/2111.06578 by Liu, Kong and @sewoong79
, or https://
arxiv.org/abs/2112.03548 by @praveshkkothari @pasin30055 @ameya_pa 5/n -
Private Algorithms Automatically Guarantee Robustness Properties
By
–
I like these works, but it would be nice to see something based solely on robustness or privacy, rather than properties of the algorithms themselves. Here's a work by @kris_georgiev1 & @Samuel_BKH that shows private algorithms are automatically robust https://
arxiv.org/abs/2211.00724 6/n -
Formalizing Robustness and Privacy in Algorithm Design
By
–
Intuitively, robustness and privacy of an algorithm are closely related: they both say the algorithm should be somehow insensitive to changing a small amount of the dataset. Formalizing connections between the two has proven to be a bit trickier. 3/n
-
Private Estimators Framework: Dwork and Lei’s 2009 Breakthrough
By
–
This direction was kicked off by Dwork and Lei in 2009, who gave a framework for designing private estimators, which works especially well given robust ones (
https://
stat.cmu.edu/~jinglei/dl09.
pdf
…). But in just the last 2-3 years, there's been a lot more interest in finding connections. 4/n -
Robustness Implies Privacy in Statistical Estimation
By
–
New paper with @Samuel_BKH
, @mahbodm_
, and Shyam Narayanan: "Robustness Implies Privacy in Statistical Estimation." A robust estimator can be (black-box) converted to a private estimator, often optimally. https://
arxiv.org/abs/2212.05015 1/n -
Hand-coding fixes to machine-learned models ultimately fail
By
–
Hand-coding fixes to massive machine-learned models always fails in the end.