Crystal Property Prediction Based On Attention Mechanism

Crystal is an indispensable and important material in modern society,and it is a hot spot in the research of modern materials science.The prediction of crystal properties is a longstanding challenge in physical materials science.Physical materials science has accumulated a large amount of crystal data in the long-term development,including experimental data and data calculated based on simulation software,which provides a basis for the application of deep learning.The spatial structure of crystals is rather special,specifically,crystals are formed by the periodic repetitive arrangement of cells in space according to a regular pattern.Due to the special spatial structure,there are three difficulties in modeling crystals: first,the need to model complex interactions between atoms within the cell;second,the need to model interaction between unit cells;and third,the fact that the properties of crystals do not change with the location in which they are located,so the designed model needs to have equivariant properties,i.e.,the predicted output of the model does not change with the translation and rotation of the crystals.The attention mechanism is a hot algorithm that has been widely used in natural language processing and computer vision with great success.The attention mechanism is good at capturing global information and is able to simulate the complex interactions between microscopic particles inside the crystal.It is innovative and feasible to use attention mechanism to predict crystal properties.In order to solve the three difficulties of crystal modeling,firstly,use the attention mechanism to model a single unit cell,and design a position encoding module specifically for crystalline materials with the special feature that the atoms in the cell have actual positions.Through a large number of experiments on the S2 S superconducting crystal dataset,the effectiveness and superiority of the attention mechanism for crystal modeling are verified.Secondly,a hierarchical attention mechanism is designed for the periodic nature of crystals,using a cell-based attention mechanism to simulate interatomic interactions within the cell and a crystal-based attention mechanism to simulate intercellular interactions.Thirdly,for crystal properties that do not change with the translation and rotation of the crystal,an equivariant attention network is designed so that the prediction output of the model will not change with the translation and rotation of the three-dimensional configuration of the crystal.Finally,by combining the hierarchical attention mechanism with the equivariant network,a periodic equivariant attention network is designed,which makes the model have equivariant properties while making full use of the periodicity of the crystal,effectively solving the three problems existing in crystal modeling.The effectiveness of the periodic equivariant attention network is verified through extensive experiments on three crystal datasets,Perov-5,MP-20,and Random-10.