Novelty Detection Based On Adversarial-transformer

Novelty detection is the identification of observations that are significantly different from the target class in the dataset.The purpose is to identify data that are not present during the detection process in the training process or that are significantly different from some aspects of the training data.As a research area with wide application value,the effectiveness of image novelty(anomaly)detection has been greatly improved with the hot development of deep learning in recent years.Detecting and identifying anomalies in images or videos is a challenging and rewarding task.The task is challenging because it is often difficult to fully obtain a priori knowledge about image anomalies in some real-world scenarios,while at the same time requiring the constructed novelty detection models to be able to learn to distinguish differences in different images under unsupervised conditions.There are also many real-world application scenarios,such as in modern smart manufacturing factories for detecting defects and flaws in products,in biomedical fields for discovering possible lesions in medical images,in traffic security fields for discovering contraband in packages,in smart security fields for detecting abnormal events in videos,etc.Inspired by Generative Adversarial Network(GAN),it provides a new idea for novelty detection algorithms based on deep learning because of its unique generator-discriminator structure: during training GAN only needs to learn the data distribution of normal samples.During testing,the input is judged to be an abnormal sample or not based on the gap between the reconstructed image and the original image.And Vision Transformer(ViT),an application of Transformer in the field of computer vision,effectively overcomes the limitation problem caused by convolutional induction bias and is more conducive to learning knowledge on data of a certain scale.Based on the above ideas,this paper proposes an adversarial training method to detect out-of-distribution samples in end-to-end trainable deep models.To this end,this paper jointly trains two deep networks R and D.The latter acts as a detector,while the former learns the probability distribution of the target class by creating adversarial examples during the training process and assists the latter in detecting the novelty class more efficiently during the testing process.In this paper,based on this idea mentioned above,we conduct the following studies on novelty detection:(1)the proposed adversarial-Transformer-based novelty detection model;(2)data augmentation of the original data to maximize the value of the data without substantially and materially increasing the number of sample data,so that the limited data can produce the value equivalent to more data to improve the model’s performance.In order to verify the performance of the above model,this paper uses the publicly available datasets MNIST and Caltech-256 for image novelty detection,while later,experiments on UMN and UCSD datasets for video frame novelty detection are conducted.The experiments show that the model proposed in this paper achieves a high level of performance on these datasets.