Domain Specific Language For Neural Networks

In recent years,the artificial neural network(ANN)has gained great attention from both academia and industry.With the emerging of neural network algorithms,the challenge of programming neural networks on different platforms has been raised.The problem of reducing the burden of programming and providing users flexible and portable programming interface under the premise of ensuring the high performance of the system has become urgently to be resolved.To address the programming challenge,in this work,we propose a novel program-ming language(NNL)and compiler specific for neural networks,which can support multiple platforms,including CPU,GPU,and ASICs.We make the following contributions in this paper.1.We propose a programming language specific for neural network algorithms(NNL)which is efficient,portable and easy to use.NNL provides high-level abstract interfaces and the explicit parallel programming.NNL constructs neural network topol-ogy with self-defined grammar,which is able to ease the programming efforts.In addi-tion,NNL provides explicit programming interfaces for processing the large amount of parallel computations in neural network algorithms.NNL is portable across platforms including CPU,GPU and ASICs.2.We propose and develop a compiler for NNL.In order to leverage the char-acteristic of neural networks,we use directed acyclic graph(DAG)as the form of the intermediate representation of the compiler,which is able to connect the front-end and the back-end of the compiler.In addition,the compiler applies two types of optimization on the block,i.e,block-fusion and block-spliting,which allow the compiler to gener-ate high-efficiency and performance-portable code by leveraging DAG transformations and graph optimization.We evaluate the NNL with mainstream neural networks on multiple computational devices,and compare the performance with existed neural network frameworks.The experiment proves that on the GPU platform,NNL is able to achieve a speedup of 2.2x on average compared to CUDA implementations,and on the ASIC platform,NNL achieves a performance rate of 0.98x compared to the ideal implementation using the assembly language.