MOCCA: multilayer one-class classification for anomaly detection
Massoli F. V., Falchi F., Kantarci A., Akti S., Ekenel H. K., Amato G.
Artificial Intelligence Computer Networks and Communications Deep learning (DL) Anomaly detection (AD) One-class (OC) classification I.5 Computer Vision and Pattern Recognition (cs.CV) FOS: Computer and information sciences Computer Science Applications Artificial Intelligence (cs.AI) 68-XX Software Computer Science - Artificial Intelligence Computer Science - Computer Vision and Pattern Recognition
Anomalies are ubiquitous in all scientific fields and can express an unexpected event due to incomplete knowledge about the data distribution or an unknown process that suddenly comes into play and distorts the observations. Usually, due to such events' rarity, to train deep learning (DL) models on the anomaly detection (AD) task, scientists only rely on "normal" data, i.e., nonanomalous samples. Thus, letting the neural network infer the distribution beneath the input data. In such a context, we propose a novel framework, named multilayer one-class classification (MOCCA), to train and test DL models on the AD task. Specifically, we applied our approach to autoencoders. A key novelty in our work stems from the explicit optimization of the intermediate representations for the task at hand. Indeed, differently from commonly used approaches that consider a neural network as a single computational block, i.e., using the output of the last layer only, MOCCA explicitly leverages the multilayer structure of deep architectures. Each layer's feature space is optimized for AD during training, while in the test phase, the deep representations extracted from the trained layers are combined to detect anomalies. With MOCCA, we split the training process into two steps. First, the autoencoder is trained on the reconstruction task only. Then, we only retain the encoder tasked with minimizing the L-2 distance between the output representation and a reference point, the anomaly-free training data centroid, at each considered layer. Subsequently, we combine the deep features extracted at the various trained layers of the encoder model to detect anomalies at inference time. To assess the performance of the models trained with MOCCA, we conduct extensive experiments on publicly available datasets, namely CIFAR10, MVTec AD, and ShanghaiTech. We show that our proposed method reaches comparable or superior performance to state-of-the-art approaches available in the literature. Finally, we provide a model analysis to give insights regarding the benefits of our training procedure.
Source: IEEE Transactions on Neural Networks and Learning Systems 33 (2021): 2313–2323. doi:10.1109/TNNLS.2021.3130074
Publisher: Institute of Electrical and Electronics Engineers, - New York, NY, USA, Stati Uniti d'America
[2] D. M. Hawkins, Identification of outliers. Springer, 1980, vol. 11. 1
[3] F. Y. Edgeworth, “Xli. on discordant observations,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, vol. 23, no. 143, pp. 364-375, 1887. 1
[4] R. Vinayakumar, M. Alazab, K. Soman, P. Poornachandran, A. AlNemrat, and S. Venkatraman, “Deep learning approach for intelligent intrusion detection system,” IEEE Access, vol. 7, pp. 41 525-41 550, 2019. 1
[5] S. Zavrak and M. I˙skefiyeli, “Anomaly-based intrusion detection from network flow features using variational autoencoder,” IEEE Access, vol. 8, pp. 108 346-108 358, 2020. 1
[6] W. Alhakami, A. ALharbi, S. Bourouis, R. Alroobaea, and N. Bouguila, “Network anomaly intrusion detection using a non-parametric bayesian approach and feature selection,” IEEE Access, vol. 7, pp. 52 181-52 190, 2019. 1
[7] M. Hasan, J. Choi, J. Neumann, A. K. Roy-Chowdhury, and L. S. Davis, “Learning temporal regularity in video sequences,” in IEEE CCVPR, 2016, pp. 733-742. 1, 8
[8] R. G. Stafford, J. Beutel et al., “Application of neural networks as an aid in medical diagnosis and general anomaly detection,” Jul. 19 1994, US Patent 5,331,550. 1
[9] T. Fernando, S. Denman, D. Ahmedt-Aristizabal, S. Sridharan, K. R. Laurens, P. Johnston, and C. Fookes, “Neural memory plasticity for medical anomaly detection,” Neural Networks, 2020. 1
[10] K. Ouardini, H. Yang, B. Unnikrishnan, M. Romain, C. Garcin, H. Zenati, J. P. Campbell, M. F. Chiang, J. Kalpathy-Cramer, V. Chandrasekhar et al., “Towards practical unsupervised anomaly detection on retinal images,” in DART. Springer, 2019, pp. 225-234. 1
[11] T. Schlegl, P. Seebo¨ck, S. M. Waldstein, U. Schmidt-Erfurth, and G. Langs, “Unsupervised anomaly detection with generative adversarial networks to guide marker discovery,” in ICIPMI. Springer, 2017, pp. 146-157. 1, 2, 6, 7
[12] A. Roy, J. Sun, R. Mahoney, L. Alonzi, S. Adams, and P. Beling, “Deep learning detecting fraud in credit card transactions,” in SIEDS. IEEE, 2018, pp. 129-134. 1
[13] A. Pumsirirat and L. Yan, “Credit card fraud detection using deep learning based on auto-encoder and restricted boltzmann machine,” IJACSA, vol. 9, no. 1, pp. 18-25, 2018. 1
[14] B. Lebichot, Y.-A. Le Borgne, L. He-Guelton, F. Oble´, and G. Bontempi, “Deep-learning domain adaptation techniques for credit cards fraud detection,” in INNSBDDL. Springer, 2019, pp. 78-88. 1
[15] U. Fiore, A. De Santis, F. Perla, P. Zanetti, and F. Palmieri, “Using generative adversarial networks for improving classification effectiveness in credit card fraud detection,” Information Sciences, vol. 479, pp. 448-455, 2019. 1
[16] K. Tout, F. Retraint, and R. Cogranne, “Automatic vision system for wheel surface inspection and monitoring,” in ASNT Annual Conference 2017, 2017, pp. 207-216. 1
[17] A. Kumar, “Computer-vision-based fabric defect detection: A survey,” IEEE TIE, vol. 55, no. 1, pp. 348-363, 2008. 1
[18] C. C. Aggarwal, “Outlier analysis,” in Data mining. Springer, 2015, pp. 237-263. 1
[19] R. Chalapathy and S. Chawla, “Deep learning for anomaly detection: A survey,” arXiv preprint arXiv:1901.03407, 2019. 1
[20] C. Zhou and R. C. Paffenroth, “Anomaly detection with robust deep autoencoders,” in Proceedings of the 23rd ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, 2017, pp. 665-674. 2
[21] D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. v. d. Hengel, “Memorizing normality to detect anomaly: Memoryaugmented deep autoencoder for unsupervised anomaly detection,” in Proc. ICCV, 2019, pp. 1705-1714. 2, 8
[22] X. Wang, Y. Du, S. Lin, P. Cui, Y. Shen, and Y. Yang, “adVAE: A self-adversarial variational autoencoder with Gaussian anomaly prior knowledge for anomaly detection,” KBS, vol. 190, p. 105187, 2020. 2
[23] T. Schlegl, P. Seebo¨ck, S. M. Waldstein, G. Langs, and U. SchmidtErfurth, “f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks,” MIA, vol. 54, pp. 30-44, 2019. 2
[24] S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon, “GANomaly: Semi-supervised anomaly detection via adversarial training,” in ACCV. Springer, 2018, pp. 622-637. 2, 7
[25] D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S.-K. Ng, “MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks,” in ICANN. Springer, 2019, pp. 703-716. 2
[26] A. Borghesi, A. Bartolini, M. Lombardi, M. Milano, and L. Benini, “A semisupervised autoencoder-based approach for anomaly detection in high performance computing systems,” Engineering Applications of Artificial Intelligence, vol. 85, pp. 634-644, 2019. 2
[27] N. Pawlowski, M. C. Lee, M. Rajchl, S. McDonagh, E. Ferrante, K. Kamnitsas, S. Cooke, S. Stevenson, A. Khetani, T. Newman et al., “Unsupervised lesion detection in brain CT using bayesian convolutional autoencoders,” 2018. 2
[28] Y. Li, X. Huang, J. Li, M. Du, and N. Zou, “SpecAE: Spectral autoencoder for anomaly detection in attributed networks,” in Proc. CIKM, 2019, pp. 2233-2236. 2
[29] K. Zhou, S. Gao, J. Cheng, Z. Gu, H. Fu, Z. Tu, J. Yang, Y. Zhao, and J. Liu, “Sparse-GAN: Sparsity-constrained generative adversarial network for anomaly detection in retinal OCT image,” in ISBI. IEEE, 2020, pp. 1227-1231. 2
[30] P. C. Ngo, A. A. Winarto, C. K. L. Kou, S. Park, F. Akram, and H. K. Lee, “Fence GAN: towards better anomaly detection,” in ICTAI. IEEE, 2019, pp. 141-148. 2
[31] M. Sabokrou, M. Khalooei, M. Fathy, and E. Adeli, “Adversarially learned one-class classifier for novelty detection,” in Proc. CVPR, 2018, pp. 3379-3388. 2
[32] J. Wang and A. Cherian, “GODS: Generalized one-class discriminative subspaces for anomaly detection,” in Proc. ICCV, 2019, pp. 8201-8211. 2
[33] E. Hong and Y. Choe, “Latent feature decentralization loss for one-class anomaly detection,” IEEE Access, vol. 8, pp. 165 658-165 669, 2020. 2
[34] I. Razzak and T. M. Khan, “One-class support tensor machines with bounded hinge loss function for anomaly detection,” in 2020 Int. Joint Conf. on Neural Networks (IJCNN). IEEE, 2020, pp. 1-8. 2
[35] H. Zenati, C. S. Foo, B. Lecouat, G. Manek, and V. R. Chandrasekhar, “Efficient GAN-based anomaly detection,” preprint arXiv:1802.06222, 2018. 2, 7
[36] P. Perera, R. Nallapati, and B. Xiang, “OCGAN: One-class novelty detection using GANs with constrained latent representations,” in CVPR, 2019. 2
[37] J. An and S. Cho, “Variational autoencoder based anomaly detection using reconstruction probability,” Special Lecture on IE, vol. 2, pp. 1- 18, 2015. 2
[38] D. Abati, A. Porrello, S. Calderara, and R. Cucchiara, “Latent space autoregression for novelty detection,” in CVPR, 2019, pp. 481-490. 2, 4, 6, 8
[39] P. Bergmann, S. Lo¨we, M. Fauser, D. Sattlegger, and C. Steger, “Improving unsupervised defect segmentation by applying structural similarity to autoencoders,” arXiv preprint arXiv:1807.02011, 2018. 2, 7
[40] C. Huang, J. Cao, F. Ye, M. Li, Y. Zhang, and C. Lu, “Inverse-transform autoencoder for anomaly detection,” preprint arXiv:1911.10676, 2019. 2, 7, 8
[41] B. Scho¨lkopf, R. Williamson, A. Smola, J. Shawe-Taylor, and J. Platt, “Support vector method for novelty detection,” NeurIPS, pp. 582-588, 2000. 2
[42] D. M. J. Tax and R. P. W. Ruin, “Support vector data description,” Machine Learning, vol. 54, pp. 45-66, 2004. 3
[43] L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Mu¨ller, and M. Kloft, “Deep one-class classification,” in Int. Conf. on Machine Learning, 2018, pp. 4393-4402. 3, 5, 6
[44] S. M. Erfani, S. Rajasegarar, S. Karunasekera, and C. Leckie, “Highdimensional and large-scale anomaly detection using a linear one-class SVM with deep learning,” PR, vol. 58, pp. 121-134, 2016. 3
[45] R. Chalapathy, A. K. Menon, and S. Chawla, “Anomaly detection using one-class neural networks,” arXiv preprint arXiv:1802.06360, 2018. 3
[46] P. Oza and V. M. Patel, “One class convolutional neural network,” IEEE Signal Processing Letters, vol. 26, no. 2, pp. 277-281, 2019. 3
[47] F. V. Massoli, F. Carrara, G. Amato, and F. Falchi, “Detection of face recognition adversarial attacks,” CVIU, p. 103103, 2020. 3
[48] N. Papernot and P. McDaniel, “Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning,” arXiv preprint arXiv:1803.04765, 2018. 3
[49] F. Carrara, F. Falchi, R. Caldelli, G. Amato, and R. Becarelli, “Adversarial image detection in deep neural networks,” Multimedia Tools and Applications, vol. 78, no. 3, pp. 2815-2835, 2019. 3
[50] A. Krizhevsky, G. Hinton et al., “The cifar-10 database,” 2009. [Online]. Available: https://www.cs.toronto.edu/ kriz/cifar.html 4, 5, 6, 7
[51] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, “MVTec AD-a comprehensive real-world dataset for unsupervised anomaly detection,” in Proc. CVPR, 2019, pp. 9592-9600. 4, 5, 6, 7, 8
[52] W. Luo, W. Liu, and S. Gao, “A revisit of sparse coding based anomaly detection in stacked RNN framework,” in Proc. ICCV, 2017, pp. 341- 349. 4, 5, 7, 8
[53] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. 5
[54] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” preprint arXiv:1312.6114, 2013. 6
[55] A. Van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves et al., “Conditional image generation with PixelCNN decoders,” in NeurIPS, 2016, pp. 4790-4798. 6
[56] D. Dehaene, O. Frigo, S. Combrexelle, and P. Eline, “Iterative energybased projection on a normal data manifold for anomaly localization,” arXiv preprint arXiv:2002.03734, 2020. 7
[57] M. Sabokrou, M. Pourreza, M. Fayyaz, R. Entezari, M. Fathy, J. Gall, and E. Adeli, “AVID: Adversarial visual irregularity detection,” in ACCV. Springer, 2018, pp. 488-505. 7
[58] F. Carrara, G. Amato, L. Brombin, F. Falchi, and C. Gennaro, “Combining GANs and autoencoders for efficient anomaly detection,” preprint arXiv:2011.08102, 2020. 7
[59] S. Venkataramanan, K.-C. Peng, R. V. Singh, and A. Mahalanobis, “Attention guided anomaly detection and localization in images,” arXiv preprint arXiv:1911.08616, 2019. 7
[60] I. Golan and R. El-Yaniv, “Deep anomaly detection using geometric transformations,” in NeurIPS, 2018, pp. 9758-9769. 7
[61] W. Luo, W. Liu, and S. Gao, “A revisit of sparse coding based anomaly detection in stacked RNN framework,” in ICCV, 2017, pp. 341-349. 8
[62] Y. Zhao, B. Deng, C. Shen, Y. Liu, H. Lu, and X.-S. Hua, “Spatiotemporal autoencoder for video anomaly detection.” New York, NY, USA: ACM MM, 2017, p. 1933-1941. 8
[63] W. Liu, W. Luo, D. Lian, and S. Gao, “Future frame prediction for anomaly detection - a new baseline,” in CVPR, 2018, pp. 6536-6545. 8
[64] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436-444, 2015. 8
Metrics
Back to previous page
@article{oai:it.cnr:prodotti:483373,
title = {MOCCA: multilayer one-class classification for anomaly detection},
author = {Massoli F. V. and Falchi F. and Kantarci A. and Akti S. and Ekenel H. K. and Amato G.},
publisher = {Institute of Electrical and Electronics Engineers, - New York, NY, USA, Stati Uniti d'America},
doi = {10.1109/tnnls.2021.3130074 and 10.48550/arxiv.2012.12111},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
volume = {33},
pages = {2313–2323},
year = {2021}
}
0000-0003-0171-4315
