Controlled Study Finds Limited Evidence for Influence-Based Machine Unlearning

Summary

The paper evaluates Deep Perturbation Learning (DPL), a method that perturbs training images and labels along influence-derived directions, in three proposed machine-unlearning roles: a direct deletion signal, a utility-preserving regularizer, and a warm start for adversarial unlearning. Because earlier evidence for weaker roles had been used to support the stronger deletion claim, the authors test each role separately under a matched protocol with exact-seed retraining baselines. An audit of the public implementation found that image directions were computed on augmented, normalized tensors but applied to raw images, while the label perturbation was smaller than float32 resolution and therefore left labels unchanged. After correcting the image pipeline, DPL failed the direct-deletion criterion on CIFAR-10 with ResNet-18 across all three paired seeds. Its utility effects changed sign across seeds, and including direction-computation time made it worse than simple warm-start baselines. A one-seed Tiny ImageNet check also did not favor DPL as a regularizer or warm start, although preprocessing inconsistencies in the released code made the direct comparison inconclusive. The results concern random instance deletion only and do not exclude influence-based methods under other deletion regimes. The authors release a role-matched evaluation protocol and an audit checklist for perturbation-based deletion claims.