A Novel Approach For Training Moral Agents Via Reinforcement Learning

With the rapid development of science and technology,artificial intelligence has been widely applied to many fields such as medical care,traffic,financial and so on.Artificial agents are playing an increasingly important role in human life,such as healthcare robots and autonomous vehicles,etc.However,when human beings enjoying the convenience brought by artificial,they also have to handle the ethical problems,for example,robots mistakenly cut humans as steel plates to death,smart speakers suggest that human users commit suicide,and driverless cars cause deaths,etc.Therefore,how to ensure that agents interact with humans properly and friendly and enable them to have the ability to comply with the basic ethical norms of humans,is an urgent need to be resolved the issue in the field of artificial intelligence.Reinforcement learning is a learning method based on trial and error.,which can assist the agent to solve the target problem according to a specific learning strategy in the process of interacting with the environment,and at the same time obtain the maximum return.In this paper,a novel reinforcement learning method for training ethical agents is proposed and completes the training of ethical intelligence based on this.Specifically,this article focuses on the following research contents:(1)A behavioral data set that can reflect the common values of human beings is constructed.The data set can train agents to obtain human values and make them abide by human ethics and ethics.Using crowdsourcing technology to collect human behavior data expressed in natural language,solves the common shortcomings of high cost,time-consuming,and bias when constructing data sets.The plot graphs and trajectory tree is generated with the help of text clustering and association analysis and other techniques,which are used to define the basic behavior space during the training of the agent and restrict the sequence of behaviors.(2)The concept of meta-ethical behavior is proposed,and nine kinds of meta-ethical behaviors are extracted based on the "Standards of Daily Behavior of Middle School Students" to realize the generalization of similar behaviors in different scenarios and at the same time expand the behavior space of the agent.We further designed an ethical grading mechanism for human behavior that considers moral,normative,and legal factors,and crowdsourcing technology is used to realize a grading mechanism of human behavior ethics.It can improve the reward and punishment mechanism in reinforcement learning so that the agent can be flexible and efficient Respond to possible human behaviors and have stronger ethical judgment capabilities.(3)By simulating common drug-buying scenarios in real life,the effectiveness and rationality of the above methods are tested.Firstly,using the Amazon Mechanical Turk crowdsourcing platform to collect the human behavior examples of this scene.Secondly,using the plot graph and trajectory tree to construct the basic behavior space of the agent training.Finally,mapping the basic behavior and meta-ethics in the drug-buying scene to the reinforcement learning environment,completing the training of the ethical agent.The experimental results show that the ethical agent obtained based on the above training method has a more significant ability to perform ethical behavior than the agent that has not been trained by the above method,indicating that the method is reasonable and effective.