Research Of Trading Strategies Based On Deep Learning And Direct Reinforcement Learning

Deep learning and direct reinforcement learning are two subdivisions of machine learning algorithms in artificial intelligence,which are widely used in the industry.While in the field of finance,quantitative risk control and AI advisor are increasingly popular,which cannot be divorced from mathematical theories and computer algorithms.However,how to design an effective network structure and quantifiable indicator,which can be migrated to financial dealings and asset allocations efficiently,is really a key factor that decides whether artificial intelligence can continue to advance in the financial field.As a type of worst-case risk measurement indicator derived from nonlinear expectation theory,G-VaR.not only satisfies the uncertainty for the mean and variance of investment income in real financial markets,but also better reflects the trading tendency of steady investors under the assumption that asset return obeys G-normal distribution.And as a typical application in nonlinear expectation,φmax-mean algorithm is used to estimate the upper and lower mean and variance of real-world sample data,whose core idea can also be migrated to the architectural design of neural networks and direct signal generation in the end-to-end trading strategy system.This thesis makes an improvement in the network structure and optimal object of AI trading system in two ways,which are deep learning and direct reinforcement learning respectively.On the one hand,in the design of end-to-end mixed models(modified CNN-DNN mixed model and modified CNN-RNN mixed model)based on deep learning,models show an advantage in mining and extraction of raw volume-price information by redefining the convolution kernel of traditional convolutional neural network.From the standpoint of empirical results,the improved CNN has an increase in both predictive accuracy and stability.In addition,based on the classical mean square error loss function in machine learning,predicted errors of asset returns are weighted and a penalty term is added for mistakes in their ordering,so we obtain two updated model optimal objects(longaware MSE and rank-aware MSE),whose performance in model training is not inferior to that of the classical MSE.On the other hand,under the direct reinforcement learning framework,inspired by the coding idea of natural language processing,we design modified CNN-RNN and modified CNN-Transformer trading signal generation systems in order to learn the relationship among multi-asset returns of the same time section.and then get the weights for asset allocation.Meanwhile.based on the core thought of φ-max-mean algorithm,we propose CNN algorithm,which is a new method for estimating the parameters in G-normal distribution.According to the test result using simulation samples,the CNN algorithm has smaller parameter estimation error than that of φ-max-mean algorithm.Subsequently,on the basis of CNN algorithm,the risk measurement indicator G-VaR derived from nonlinear expectation theory is integrated into the network structure of direct reinforcement learning system as a penalty term for the performance function.The back test result of CSI500 reinforced strategy shows that,compared with benchmark CSI500 and the trading system whose performance function is total profit or sharpe ratio,the direct reinforcement learning system employing G-VaR has a more steady investment style.Therefore,the risk measurement indicator based on the nonlinear expectation theory framework,has profound meaning and much broader prospects when combined with artificial intelligence algorithms both in the area of investment decision making and risk management.