Portfolio Management Based On Deep Reinforcemrnt Learning

Portfolio management is an important research direction in traditional finance and quantitative finance.How to dynamically redistribute funds to obtain higher returns under risk control is the key to portfolio optimization.Due to the continuous and unstable characteristics of financial markets,portfolio optimization is often ineffective when applied to traditional models.Deep reinforcement learning algorithm has gradually attracted people’s attention due to its ability to effectively establish relevant models for complex markets and dynamically manage investment portfolios in the process of exploration.It has become a hot research direction.This paper builds a portfolio management model based on three algorithms in deep reinforcement learning:Deep Deterministic Policy Gradient(DDPG)algorithm,Advantage Actor-Critic(A2C)algorithm and Proximal Policy Optimization(PPO)algorithm.Combined with Generative Adversarial Network(GAN),it optimizes portfolio management and achieves more returns under the premise of controlling risks.The main contents are as follows:(1)Construct a portfolio optimization model based on deep reinforcement learning.A new design method of reward function is proposed,which takes the change value of Sharpe ratio as the reward of action.Three deep reinforcement learning algorithms were used to explore the established environment.Nine broad-based ETFs in domestic A-share market were selected as risk assets,and their daily-frequency data were trained and tested.ETF historical trading data and 8 calculated technical indicators are used as inputs,and DDPG,A2C and PPO algorithms are used to study and explore them respectively to obtain the best trading strategy.The results obtained from the experimental backtest were compared and analyzed from the four evaluation indexes of annual return rate,sharpe ratio,max draw down and annual stability,and compared with the equal proportion holding benchmark.It was concluded that the portfolio management model based on three algorithms had significant effects in risk control and return performance,among which A2C algorithm and DDPG algorithm had better performance,The benchmark annualized yield improved by about 15%and the Sharpe ratio improved by more than 0.6.(2)The Generative Adversarial Network is added to the portfolio management model based on deep reinforcement learning,and the data is first learned through the GAN to generate synthetic data,which is used to explore the deep reinforcement learning strategy and improve the stability performance of the model.ETF data and three kinds of deep reinforcement learning algorithm models were used for backtest experiment analysis,and the pros and cons of several algorithm results were compared horizontally,and the effectiveness of the strategy was verified by vertical comparison,and the annual volatility decreases by about 0.03.(3)A portfolio management system based on deep reinforcement learning and generative adversarial network is developed.Using HTML5 combined with JavaScript,MySQL and other languages to complete the construction of each module of the system on the web side,making the visualization and operation level of the model more convenient and concise,and investors can use this system for auxiliary analysis.The portfolio model proposed in this paper shows a good performance in the stock market.The application of deep reinforcement learning algorithm and GAN to the stock market has research significance and reference value,which is conducive to the related algorithms to carry out more in-depth research in the field of quantitative finance.