Research On Stock Market Trading Based On Deep Reinforcement Learning

In the financial field,stock trading is a very important area.Constructing stock trading strategies to achieve high returns is one of the issues that investors are most concerned about.With the continuous development of computer technology,the application of artificial intelli-gence in the area of stock trading has received increasing attention.As an adaptive machine learning method,Deep Reinforcement Learning has received more and more attention in the area of stock trading.Our work is as follows:Firstly,this paper constructs a Partially Observable Markov Decision Processes(POMDP)for stock trading.Use the daily data of stocks and financial indicators to build a trading envi-ronment,We combines Long Short Term Memory(LSTM)to improve the Advantage Actor Critical(A2C)algorithm.Compared to previous studies that only used ordinary A2C,adding LSTM can get information in stock data time series,obtain hidden states in the trading en-vironment,and formulate better trading strategies.In a further experiment,30 stocks were selected from the Dow Jones Industrial Index or the S&P 500 Index as stock pools.Results show that the trading strategy based on our method outperforms the trading strategy based on the ordinary A2C algorithm and corresponding index in terms of cumulative returns and Sharpe ratio,not only in long-term trading but also in rising,fluctuating or declining market performance.It has been numerically proven that using POMDP to model the market envi-ronment and learning trading strategies using the A2C algorithm with LSTM can effectively improve returns.Secondly,in order to solve the problems of repeated training caused by stock pool changes and limited data of newly issued stocks in stock trading,this paper uses Meta Reinforcement Learning algorithm RL~2to solve problems.The advantage of Meta Reinforcement Learning algorithms is that it can learn a common model from other stock data,known as a meta model.meta model can be used to migrate experience to new portfolios or new stocks.In this part of the experiment,the S&P 500 index component stock data was used to train the meta model,and three experiments were set up to verify.The first group of experiments used stock data from the same time period as the meta model for backtest,the second group of experiments used stock data one year earlier than the meta model for backtest,and the third group of exper-iments used newly issued stocks for backtest.Results show that the trading strategy based on RL~2outperforms the stochastic model and the S&P 500 index in terms of cumulative returns and Sharpe ratios.Experiments have shown that meta reinforcement learning algorithms can effectively transfer learned experiences into new stock portfolios and achieve high cumulative returns.In summary,this article mainly completes two tasks.One is to use POMDP to model the stock trading market,and combine LSTM and A2C algorithms,to enhance the ability to process time series and get hidden states in the stock market.Experiments were conducted on the Dow Jones Industrial Index and the S&P 500 component stocks,respectively,and numer-ical results showed that the trading strategies learned by our method can effectively improve returns.The other is to use Meta Reinforcement Learning algorithm to solve the problems of repeated training due to changes in the stock pool and limited data of newly issued stocks.Three experiments were conducted to numerically verify that the trained meta model can ef-fectively transfer experience and learn excellent trading strategies.