Name Stability Training

Intent

Motivation

Structure

<Diagram>

Discussion

Stability training. It is straightforward to implement stability training for any given neural network by adding a Gaussian perturbation sampler to generate perturbed copies of the input image x and an additional stability objective layer.

Known Uses

Related Patterns

<Diagram>

References

http://arxiv.org/pdf/1604.04326v1.pdf Improving the Robustness of Deep Neural Networks via Stability Training

Stability training makes the output of a neural network more robust by training a model to be constant on images that are copies of the input image with small perturbations. As such, our method can enable higher performance on noisy visual data than a network without stability training.

The performance of stabilized models is significantly more robust for near-duplicate detection, similar image ranking and classification on noisy datasets.

http://www.kdnuggets.com/2015/07/deep-learning-adversarial-examples-misconceptions.html

https://arxiv.org/pdf/1604.04004.pdf Understanding How Image Quality Affects Deep Neural Networks The networks are surprisingly resilient to JPEG and JPEG2000 compression distortions. It is only at very low quality levels (quality parameter less than 10 for JPEG and PSNR less than 30 for JPEG2000) that the performance begins to decrease. This means that we can be reasonably confident that deep networks will perform well on compressed data, given that the compression level is sufficient.

https://arxiv.org/abs/1606.04435 Adversarial Perturbations Against Deep Neural Networks for Malware Classification

Deep neural networks, like many other machine learning models, have recently been shown to lack robustness against adversarially crafted inputs. These inputs are derived from regular inputs by minor yet carefully selected perturbations that deceive machine learning models into desired misclassifications. Existing work in this emerging field was largely specific to the domain of image classification, since the high-entropy of images can be conveniently manipulated without changing the images' overall visual appearance. Yet, it remains unclear how such attacks translate to more security-sensitive applications such as malware detection - which may pose significant challenges in sample generation and arguably grave consequences for failure. In this paper, we show how to construct highly-effective adversarial sample crafting attacks for neural networks used as malware classifiers. The application domain of malware classification introduces additional constraints in the adversarial sample crafting problem when compared to the computer vision domain: (i) continuous, differentiable input domains are replaced by discrete, often binary inputs; and (ii) the loose condition of leaving visual appearance unchanged is replaced by requiring equivalent functional behavior. We demonstrate the feasibility of these attacks on many different instances of malware classifiers that we trained using the DREBIN Android malware data set. We furthermore evaluate to which extent potential defensive mechanisms against adversarial crafting can be leveraged to the setting of malware classification. While feature reduction did not prove to have a positive impact, distillation and re-training on adversarially crafted samples show promising results.

http://www-etud.iro.umontreal.ca/~wardefar/publications/adv_perturb_chapter.pdf Perturbation, Optimization and Statistics

Simultaneously, the adversarial perspective can be fruitfully leveraged for tasks other than simple supervised learning. While the focus of generative modeling in the past has often been on models that directly optimize likelihood, many application domains express a need for realistic synthesis, including the generation of speech waveforms, image and video inpainting and super-resolution, the procedural generation of video game assets, and forward prediction in model-based reinforcement learning. Recent work (Theis et al., 2015) suggests that these goals may be at odds with this likelihoodcentric paradigm. Generative adversarial networks and their extensions provide one avenue attack on these difficult synthesis problems with an intuitively appealing approach: to learn to generate convincingly, aim to fool a motivated adversary. An important avenue for future research concerns the quantitative evaluation of generative models intended for synthesis; particular desiderata include generic, widely applicable evaluation procedures which nonetheless can be made to respect domain-specific notions of similarity and verisimilitude.

https://arxiv.org/abs/1511.04508 Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks

Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 10^30. We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.