Portfolio Optimization Based On Deep Reinforcement Learning

Portfolio problem is a common problem in financial management.Investors constantly redistribute their assets to different products in accordance with a certain proportion,and at the same time ensure higher returns for investors under the condition of controlling risks,and pay more attention to selecting the optimal investment proportion or optimal portfolio size.Given the continuity and instability of financial markets,an investor-friendly portfolio approach is often difficult to achieve.With the exploration and practical verification in the financial field,the application of deep reinforcement learning to portfolio is a topic worthy of attention.In order to achieve maximum returns for investors or enterprises under the condition of balancing returns and controlling risks,this paper adopts deep deterministic strategy gradient(DDPG)algorithm and double delay deep deterministic strategy gradient(TD3)algorithm.Deep reinforcement learning portfolio model based on CNN network and deep reinforcement learning portfolio model based on LSTM network are constructed to optimize stock portfolio management.And define markov decision process model to optimize the portfolio strategy,complete the stock portfolio transaction,and achieve the maximum return of stock investment.The main research contents are as follows:(1)A portfolio optimization model based on DDPG algorithm is constructed.The parameter space noise is added to DDPG algorithm to make the strategy exploration more complete.Five stocks in THE NASDAQ 100 index were selected as risk assets,and the data were divided into training sets and test sets for experimental analysis.The state data matrix with historical stock characteristics is taken as the input,and the features in the state data are extracted by deep neural network.Based on the features,the strategy optimization method of reinforcement learning algorithm is used to obtain the weight of the stock portfolio,and the optimal trading strategy is obtained in an exploratory way.The deep reinforcement learning portfolio model based on CNN network and the deep reinforcement learning portfolio model based on LSTM network were constructed for experimental analysis.In the case of different historical window days,the selected stocks are analyzed from the cumulative return,Sharpe ratio and maximum redrawback evaluation index.It is concluded that the deep reinforcement learning portfolio strategy model based on CNN network has better effect in balancing risk and return,and can be well applied to stock portfolio.(2)A portfolio optimization model based on TD3 algorithm is constructed.Two improvements are made on the basis of DDPG algorithm,that is,increase the number of neural networks and delay update,which increases the stability of the algorithm.A deep reinforcement learning portfolio model based on LSTM network is constructed.On this basis,six stocks were selected for experimental design and analysis.In order to improve the generalization ability of the model,three groups of experiments were set up to analyze the stock portfolio in different time periods and with different types of data to verify the effectiveness of the strategy.The portfolio strategy proposed in this paper can be effectively applied to the stock market,and the application of deep reinforcement learning to stock investors has effective reference value and significance,which is conducive to further research of deep reinforcement learning algorithm in the financial field.