This is an old revision of the document!

aka outlier detection An, Jinwon, and Sungzoon Cho. “Variational Autoencoder based Anomaly Detection using Reconstruction Probability.” (2015). Berniker, Max, and Konrad P. Kording. “Deep networks for motor control functions.” Frontiers in computational neuroscience 9 (2015). Domingues, Rémi. “Machine Learning for Unsupervised Fraud Detection.” (2015). Hooi, Bryan, et al. “BIRDNEST: Bayesian Inference for Ratings-Fraud Detection.” arXiv preprint arXiv:1511.06030 (2015). Sölch, Maximilian, et al. “Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series.” arXiv preprint arXiv:1602.07109 (2016). Xie, Jianwen, et al. “A theory of generative convnet.” arXiv preprint arXiv:1602.03264 (2016). Universum Prescription: Regularization using Unlabeled Data

This paper shows that simply prescribing “none of the above” labels to unlabeled data has a beneficial regularization effect to supervised learning. We call it universum prescription by the fact that the prescribed labels cannot be one of the supervised labels. In spite of its simplicity, universum prescription obtained competitive results in training deep convolutional networks for CIFAR-10, CIFAR-100, STL-10 and ImageNet datasets. A qualitative justification of these approaches using Rademacher complexity is presented. The effect of a regularization parameter – probability of sampling from unlabeled data – is also studied empirically. A recommender system for efficient discovery of new anomalies in large-scale access logs (Note: Not Deep Learning)

Our system Helios discovers and recommends unknown and unseen anomalies in large-scale access logs with minimal supervision and no starting information on users and items. Typical recommender systems assume availability of user-and item-related information, but such information is not usually available in access logs. To resolve this problem, we first use discrete categorical labels to construct categorical combinations from access logs in a bootstrapping manner. Then, we utilize rank statistics of entity rank and order categorical combinations for recommendation. From a double-sided cold start, with minimal supervision, Helios learns to recommend most salient anomalies at large-scale, and provides visualizations to security experts to explain rationale behind the recommendations.

Discover: Generate Baseline - Details

Rank & Recommend - Details As a discovery and recommender system Meta-Unsupervised-Learning: A supervised approach to unsupervised learning LEARNING TO REMEMBER RARE EVENTS

Despite recent advances, memory-augmented deep neural networks are still limited when it comes to life-long and one-shot learning, especially in remembering rare events. We present a large-scale life-long memory module for use in deep learning. The module exploits fast nearest-neighbor algorithms for efficiency and thus scales to large memory sizes. Except for the nearest-neighbor query, the module is fully differentiable and trained end-to-end with no extra supervision. It operates in a life-long manner, i.e., without the need to reset it during training. Our memory module can be easily added to any part of a supervised neural network. To show its versatility we add it to a number of networks, from simple convolutional ones tested on image classification to deep sequence-to-sequence and recurrent-convolutional models. In all cases, the enhanced network gains the ability to remember and do life-long one-shot learning. Our module remembers training examples shown many thousands of steps in the past and it can successfully generalize from them. We set new state-of-the-art for one-shot learning on the Omniglot dataset and demonstrate, for the first time, life-long one-shot learning in recurrent neural networks on a large-scale machine translation task.

Recurrent generative auto-encoders and novelty search Detection of Anomalies in Large Scale Accounting Data using Deep Autoencoder Networks

To overcome this challenge we propose a novel method of detecting anomalous journal entries using deep autoencoder networks. We demonstrate that the trained networks’ reconstruction error regularized by the individual attribute probabilities of a journal entry can be interpreted as a highly adaptive anomaly assessment. Our empirical study, based on two datasets of real-world journal entries, demonstrates the effectiveness of the approach and outperforms several baseline anomaly detection methods. Resulting in a fraction of less than 0.15% (0.7%) of detected anomalous entries while achieving a high detection precision of 19.71% (9.26%). Initial feedback received by accountants underpinned the quality of our approach capturing highly relevant anomalies in the data. We envision this method as an important supplement to the forensic examiners’ toolbox. Metric Learning for Novelty and Anomaly Detection

We show that metric learning provides a better output embedding space to detect data outside the learned distribution than cross-entropy softmax based models. This opens an opportunity to further research on how this embedding space should be learned, with restrictions that could further improve the field. The presented results suggest that out-of-distribution data might not all be seen as a single type of anomaly, but instead a continuous representation between novelty and anomaly data. In that spectrum, anomaly detection is the easier task, giving more focus at the difficulty of novelty detection. Learning Confidence for Out-of-Distribution Detection in Neural Networks