Research Of Categorization Modeling Of Reinforcement Learning

Traditional reinforcement learning framework is based on the MDP(Markov Decision Process).However,the actual environment of agent is often complex,and the traditional model may not be convenient to use.So many new models have been created to meet different practical needs,such as distributed reinforcement learning,multi-person reinforcement learning,hierarchical reinforcement learning theory and so on.In order to better display the respective structural characteristics and essence of each model,as well as the relationship between each model.We propose to generalize,algebraize and categorize the reinforcement learning model using the category theory widely used in modern mathematics.Category models in category theory can represent complex mathematical abstract structures well.By abstracting the reinforcement learning model into a category model,we can put many analysis and modeling tools in modern mathematics into reinforcement learning,and strictly algebraic reinforcement learning,so as to more clearly study the theoretical mathematical basis of relevant problems and the essential abstract structure from the perspective of computation,and get rid of the inconveniences and ambiguities caused by some non-general and non-strict terms and constructions in various practical models constructed by different schools of scholars in the traditional field of reinforcement learning.By abstracting the actions and strategies of reinforcement learning into the combinatorial extensible morphisms of category theory,we can construct and model huge practical reinforcement learning models conveniently.By modeling a variety of models into specific categories,we can study the correlation functors among these categories and the natural transformations between these functors to study the relationship between the models,and by constructing new categories,functors and natural transformation,we can construct some more general reinforcement learning models.By generalizing some basic concepts required by reinforcement learning model into basic abstract concepts in category theory,this paper successfully modeled the state transition process of perfect information as well as imperfect information reinforcement learning under deterministic and non-deterministic conditions as related categories.Moreover,by introducing the lens theory from Haskell community,we successfully introduced the reward feedback mechanism of the actual reinforcement learning process with reward signals into relevant categories.Although this is just an attempt of modeling the basic model of reinforcement learning,but it can give a lot of other complex reinforcement learning model for reference,including promote fusion more easily with research achievements in other field of decision-making,especially with the field of game theory that reinforcement learning field is closely related to.