Neural Network Based On Attention Mechanism: Feature Selection Using Cognitive Feedback

Artificial Neural Network research has got great progress in many pattern recognition problems, but they are still not competent for multiple objects recognition and segmentation task in complex scene until now. Neuroscience research in attention mechanism has pointed out a possible solution for this problem, but there is still not any model based on this mechanism have got good performance in real-world data. The goal of this paper is building a neural network model based on attention mechanism for multiple objects recognition and segmentation problem in complex scene, and proveing its competence in real world data. For this goal, we developed a new neural network model named Attentional Neural Network(a NN). Attentional Neural Network is a new framework that integrates top-down cognitive bias and bottom-up feature extraction in a coherent architecture. The top-down influence is especially effective when dealing with high noise or difficult segmentation problems. It works with feature extracting process used by general feed-forward neural network, constructed a entirety which can select, segment and recognize objects.Our system is modular and extensible. It is also easy to train and cheap to run, and yet can accommodate complex behaviors. We obtain classification accuracy better than or competitive with state of art results on the MNIST variation dataset of hand writing digits. In white noise background dataset MNIST-background-rand, we got error rate of 3.22%, better than any existing methods; in natural image background dataset MNIST-background-image, we got error rate of 15.33%, competitive with existing best method. This model also got great performance in harder task of overlapping digit segmentation. It successfully disentangled at least one overlaid digits with a high success rates of 95.46%, competitive with human being performance. We view such a general purpose framework as an essential foundation for a larger system integrating different cognitive abilities such as recognition, segmentation and tracing. In summary, we looked into the future of solve many different patternrecognition problems and its possibility of practical engineering.