SAFETY - AI Dynamics

Camouflage Poisoning Attack on Machine Learning Models

By

–

13 January 2023 12h58

For more details, please check out the paper and the code! And definitely remember the names of the first authors, Waterloo undergrads Jimmy Di (applied to grad schools this year) and Jack Douglas. https://
arxiv.org/abs/2212.10717 https://
github.com/Jimmy-di/camou
flage-poisoning
… 10/10

→ View original post on X — @thegautamkamath

13 January 2023

Defending Against Data Poisoning Through Strategic Point Addition

By

@thegautamkamath

–

13 January 2023 12h58

From a technical perspective, we raise an interesting new question. Usually, one mitigates data poisoning attacks by *removing* training points. But camouflage essentially asks: can you negate data poisoning attacks by *adding* points! Interesting beyond MU. 7/n

→ View original post on X — @thegautamkamath

13 January 2023

Adversarial Attacks on Machine Learning Model Updates Research

By

@thegautamkamath

–

13 January 2023 12h58

We were inspired by previous papers in the "attacking MU" space. See https://
arxiv.org/abs/2109.08266 by Marchant @bipr @ScottAlfeld
, which poisons MU to make it take longer. 8/n

→ View original post on X — @thegautamkamath

13 January 2023

Dataset Poisoning Attack via Malicious Machine Unlearning

By

@thegautamkamath

–

13 January 2023 12h58

Here's how it works. The adversary first poisons the dataset, and the victim trains a model on it. Everything behaves as normal. But the adversary later requests some of their points to be unlearned. Only after the unlearning, then the model behaves in some malicious way. 4/n

→ View original post on X — @thegautamkamath

13 January 2023

Targeted Poisoning Attacks in Machine Learning Models

By

@thegautamkamath

–

13 January 2023 12h58

In this work we focus on targeted poisoning attacks (the goal is to misclassify a specific point in the test set) & used this attack (
https://
arxiv.org/abs/2009.02276) ft @jonasgeiping @wronnyhuang @tomgoldsteincs
. But ours is a proof of concept, neither is intrinsic to the framework. 5/n

→ View original post on X — @thegautamkamath

13 January 2023

Camouflage Data Poisoning Attacks in ML Unlearning Systems

By

@thegautamkamath

–

13 January 2023 12h58

But when you add something new to an ML pipeline, there's new ways for adversaries to wreak havoc. We introduce a new type of data poisoning attack exploiting the dynamic nature of unlearn requests: a "camouflage" attack, which lies dormant until triggered by the adversary. 3/n

→ View original post on X — @thegautamkamath

13 January 2023

Machine Unlearning Vulnerability Enables Poisoning Attacks

By

@thegautamkamath

–

13 January 2023 12h58

New paper led by Jimmy Di & Jack Douglas, co-advised with @AcharyaJayadev @ayush_sekhari
: "Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks" The adaptive nature of machine unlearning opens a new attack vector for adversaries. https://
arxiv.org/abs/2212.10717 1/n

→ View original post on X — @thegautamkamath

13 January 2023

DeepMind CEO hints at 2023 Sparrow beta release citing sources

By

@aibreakfast

–

13 January 2023 1h37

CEO of Google's DeepMind @demishassabis hinted today at a 2023 private beta release of "Sparrow" (DeepMind's version of ChatGPT) that will be capable of citing sources for its responses.

→ View original post on X — @aibreakfast

13 January 2023