AI
Why Privacy-Preserving ML Exists and Where Learnable Obfuscation Fits
Three hospitals each hold patient X-rays. None can legally share them. Together, they have enough data to train a tumor detector. That tension, multiplied by GDPR and every other multi-party data constraint, is what drove privacy-preserving ML into existence. Provable methods like homomorphic encryption and secure multi-party computation solve the problem mathematically but run too slowly for real workloads. Learnable obfuscation trades that guarantee for speed, transforming data with masking, mixing, or permutation before it ever leaves a party’s hands. The cost is formal security proof.
“the current research in landscape it’s actually a game. I design a method you break it. I design another method somebody else break it and so on.” — Mihail-Iulian Pleșa
The Three Techniques of Learnable Obfuscation: Masking, Mixing, and Permutation
Learnable obfuscation uses three primitives. Masking adds the same random image to every sample before training, turning an eight into noise while preserving enough structure for a model to learn. Mixing creates weighted combinations — a 70% dog, 30% cat image with a soft label to match — so no single original is ever exposed. Permutation rotates pixel rows by a fixed offset and is usually applied on top of one of the other two. All parties agree on the same transformation upfront, apply it locally, then upload only the obfuscated data. The model trains on that shared pool; each party downloads the result and runs inference on its own machine, never touching plaintext in the cloud.
The Security Gap: Why Learnable Obfuscation Cannot Match Cryptographic Guarantees
Cryptographic security has a precise definition: given two ciphertexts, an attacker cannot tell which plaintext each hides, even after querying an encryption oracle thousands of times. Learnable obfuscation cannot meet that bar. An attacker watching a training run sees 99% accuracy and deduces the data is simple MNIST digits, without touching a raw image. That signal alone breaks indistinguishability.
“the current research in landscape it’s actually a game. I design a method you break it. I design another method somebody else break it and so on.” — Mihail-Iulian Pleșa
Narrower goals remain possible: block reconstruction, resist membership inference, prevent reverse engineering. A 2024 paper combined masking, mixing, and permutation and proved reconstruction fails below a specific plaintext–transform pair threshold. No method yet covers all three goals.
NeuroCrypt: A Provably Secure Masking Scheme That Is Practically Broken
NeuroCrypt applies the same random neural network to every image in a dataset. The NeuroCrypt MNIST transform demo (27:21) shows why that looks safe: transformed MNIST digits appear as pure noise. The scheme has an information-theoretic proof. The flaw is in what the proof assumes.
the theorem is correct the proof works but in reality this system is not secure because when they formalize the security and they prove the security they didn’t take into account that an attacker might obtain a small a very small subset of transformed data. — Mihail-Iulian Pleșa
The NeuroCrypt reconstruction attack with 50 pairs (32:22) proves it. Against a 60,000-image MNIST dataset, 50 plaintext-transform pairs are enough. An attacker trains a neural network on that tiny subset and recovers recognizable digits across the full set. Applying the same shared network to all images gives the attacker one small lever to move everything.
InstaHide: A Mixing Scheme Undone by Public Weights
InstaHide mixes images as weighted sums and flips pixel signs. The mixing demo on CIFAR-10 combines a car (34%) and a bird (66%) into noise, yet a classifier trained on such mixtures hits 80% accuracy. The paper’s proof ties breaking InstaHide to the subset-sum problem. But it assumes the mixing weights are secret. They are not: soft training labels expose them directly (58% airplane, 42% bird).
“it’s uh actually held high school maths” — Mihail-Iulian Pleșa
The linear-system reconstruction attack treats each source image as an unknown and solves for it using the leaked weights. No neural network required. Sign flips collapse by taking absolute values. Recovered images are recognizable.
What to Use Now and Where the Field Is Heading
Learnable obfuscation runs on weighted sums and random neural networks, not expensive cryptography. That efficiency is real. The security is not. NeuroCrypt and InstaHide each shipped a proof, and each proof failed once the attacker held a small sample of plain-and-transformed pairs. Both proofs assumed that could never happen. It can, and does.
“right now I won’t be able to convince anyone to use this type of methods.” — Mihail-Iulian Pleșa
For production today, secure multi-party computation is the safer choice. Communication overhead is high but manageable for small datasets. Learnable obfuscation may catch up: a 2024 paper formally proved reconstruction resistance under a sample-count threshold. That is progress. Until the security definitions cover membership inference and reverse engineering too, treat these methods as research, not infrastructure.
Q&A
In which situations could you convince a data owner to use learnable obfuscation, given how easy it is to attack? The speaker said flatly that he cannot convince anyone to use learnable obfuscation today, and recommended secure multi-party computation instead for small datasets, while pointing to a recent paper as grounds for cautious future optimism. ▶ 40:54
In the NeuroCrypt demo, if you increase the attacker’s sample count beyond 50, does reconstruction quality improve? Yes — the speaker demonstrated live by raising the sample count from 50 to 100 and then 500, showing visibly cleaner digit reconstructions at each step. ▶ 42:13
Does learnable obfuscation still work for high-stakes medical imaging like MRI tumor detection, or does it damage accuracy? It does not hurt accuracy — the speaker said all learnable obfuscation papers test across data complexities from MNIST to ImageNet specifically to show that accuracy holds, with the tradeoff being a larger model and more training epochs rather than lost features. ▶ 43:10
Notable Quotes
the current research in landscape it’s actually a game. I design a method you break it. I design another method somebody else break it and so on. Mihail-Iulian Pleșa · ▶ 4:49
the theorem is correct the proof works but in reality this system is not secure because when they formalize the security and they prove the security they didn’t take into account that an attacker might obtain a small a very small subset of transformed data. Mihail-Iulian Pleșa · ▶ 21:12
it’s uh actually held high school maths Mihail-Iulian Pleșa · ▶ 36:50
right now I won’t be able to convince anyone to use this type of methods. Mihail-Iulian Pleșa · ▶ 41:21
Key Takeaways
- Formal security proofs for learnable obfuscation are only as strong as their threat model assumptions — verify those assumptions before trusting the proof.
- Sharing even a small number of plaintext–transform pairs (as few as 50) is enough to break masking-based schemes like NeuroCrypt.
- Mixing weights leaked through soft training labels turn InstaHide’s hardness-based security into a solvable high-school linear algebra problem.
About the Speaker
Mihail-Iulian Pleșa
Mihail is a security researcher at Orange Services and a PhD student at the University of Bucharest. His area of research is systems security, with a focus on applied cryptography and machine learning. Mihail has authored and co-authored various research papers on topics ranging from quantum cryptography to confidential computing in cybersecurity.