Mutational Features Of Negative Reinforcement Learning In Mice And Its Basal Ganglia Mechanisms

Recently,increasing researches focused on the dynamic process of learning.Although many researchers tried to detail the learning process to find the underlying neuronal and biochemical or molecular mechanisms,few had noticed that there has been a persistent controversy about the basic characteristics of the learning process.Whether the learning process changes gradually or abruptly determines the methods used in the study about the learning process.In the last century,the learning/study field has conducted a lot of discussions on the gradual change and sudden change of the learning process,and Thorndike’s view of progressive learning and Kohler’s insight learning theory are the most prominent.However,this controversy gradually got out of the public’s view with the development of cognitive psychology,and "gradual" was regarded as the basic feature of the learning process,while insight was changed into a special kind of creativity and forgotten by mainstream learning researches,until Gallistel,Fairhurst,and Balsam published their articles which proved that the averaging analysis mislead the results on gradual/abrupt change of the learning process and pointed out that the learning process is not gradual but abrupt on PNAS in 2004.However,this important research did not attract much attention in related fields.There are many reasons for it,while one of the most important is that Gallistel’s research focused on the individual analysis of abrupt change during learning and did not propose a method to make the individual characteristic into a group characteristic,which resulted in the difficulty in researching the related mechanisms.Based on the past graduality/insight theory,this research proves the suddenness in the learning process by observing the process of negative reinforcement learning in a large number of mice,described the quantitative definition of individual mutation characteristics,and found the characteristics of its group distribution and related parameters.Based on it,this research further explored the related basal ganglia mechanisms under physiological conditions and observed the performance of the negative reinforcement learning under relevant neuropathological conditions.The main contents and results of this research are as follows:（1）Negative reinforcement learning in mice shows abrupt change characteristics among the groupFirstly,by observing a large number of negative reinforcement learning processes in mice individually,and combing with classical insight and connection learning theory,the study has qualitatively and quantitatively described the abrupt change characteristics of negative reinforcement learning in mice,and proves two factor to describe the characteristics of change:change point and change periods.The change point reflect that mice no longer fail suddenly during the learning process,and the change period reflects the abrupt decrease in response time during the learning process.The quantitative results show that the learning curve by the traditional average analysis will conceal the abrupt change characteristics of learning,and the learning curve aligned by the change point will show the abrupt change characteristics in the group.In addition,the trials and time of change point plateau period（the duration before the change point occurs）and the change period shows a positive skew distribution,which further indicates the abrupt change is a general characteristic of the negative reinforcement learning.（2）The dynamic activities of dopamine neurons in the substantia nigra represent the negative reinforcement learning process and its abrupt change characteristicsTo further investigate the dopamine mechanism of change point,the study used calcium imaging technology to observe and analyze the activity of dopamine neurons in the substantia nigra of mice during negative reinforcement learning.The results showed that the activity of dopamine showed an obvious bimodal pattern with three stage which was divide by change point during the process of negative reinforcement learning.One of the bimodal peaks is "a crest" which induced at the beginning of the electric shock,and the another is "b crest" which induced at the end of the electric shock.Before the change point,the bimodal crests of dopamine neurons calcium signal show a large "a crest" and a small"b crest",while during the duration after the change point happened and before the behavioral response time drops to within 10s,the bimodal crests of dopamine neurons calcium signal show a decreased"a crest" and a increased large "b crest".After the behavioral response time drops to within 10s,the bimodal crests of dopamine neurons calcium signal appear as a unimodal "a crest" without obvious "b crest".Nevertheless,looking at it in the round,the"b crest"in the last stage may not completely disappear,but appeared in advance.These results indicate that the establishment of negative reinforcement learning and the formation of its abrupt change characteristics are closely related to the activity of substantia nigra dopamine neurons.（3）Direct pathways and indirect pathways in DMS are involved in the formation of negative reinforcement learning abrupt change characteristicsFurther,the study explored the role of the direct pathway and indirect pathway in DMS which receive the signal from dopamine neurons in SNc in the formation of change point during negative reinforcement learning.Through microinjection and chemical genetic intervention methods,the study observes the negative reinforcement learning process under four states:the activation and inhibition of the direct pathway neurons in DMS,and the disinhibition and inhibition of the indirect pathway neurons in DMS.The results showed that,except the inhibition of indirect pathways,the mice under the other three states showed an increased average response time and disappearance or delay of change point during the negative reinforcement learning under the other three states.These results indicate that the normal activities of the direct and indirect pathways in DMS cooperate to mediate the formation of abrupt change in negative reinforcement learning.Combined with the activity pattern of dopamine neurons in the second phase,the inhibitory state of the indirect pathways may be beneficial to the formation of abrupt change.（4）The endocannabinoid system participates in the formation of abrupt change in negative reinforcement learningFurther,the study explored the role of the endocannabinoid system in the negative reinforcement learning.The results showed that both intraperitoneal and DMS microinjection of HU210 induced an increased average response time and disappearance or delay of change point during negative reinforcement learning.Intraperitoneal injection of HU210 combined with DMS microinjection of AM281 can alleviate the deficits of learning induced by HU210,while activation of D1 neurons in DMS by the chemical genetic method can completely rescue the deficits induced by HU210.These results indicate that the CB1 receptor in DMS plays a role in the direct pathway mediating the negative reinforcement learning and its abrupt change,and the CB1 receptor in the structure projecting into the DMS may also involved in the formation of abrupt change.（5）Autism-like mice induced by VPA prenatal exposure showed abnormal negative reinforcement learning and extinctionThe study further observed the characteristics of negative reinforcement learning in autism-like mice induced by VPA prenatal exposure,which was reported to have neurological disease state of the dopamine system.The results showed that prenatal exposure to VPA resulted in male offspring showing obvious autism-like behaviors,while the female offspring had no obvious autism-like behaviors;similarly,the male offspring showed difficulties in establishing learning connections during the three-day negative reinforcement learning,showed no abrupt change during the first day of learning and the increased mistakes during the last two days of learning,while the female offspring learned normally.Nevertheless,the male offspring basically established the connection between behavior and reinforcer stimulus after three days of learning and can complete the tasks in most trials.In the three-day extinction experiment,the male offspring showed an accelerated extinction than the vehicle group,while the female mice still behaved normally.In summary,the abrupt change characteristics of negative reinforcement learning can be quantitatively analyzed through the change point and the change period,and related mechanism studies can go on by using the method.The substantia nigra dopamine neurons characterize the learning change process by dynamic activities,and the D1 and D2 neurons in DMS which receive the dopamine projection from SNc cooperate to regulate the formation of learning behavior.In addition,cannabinoid receptors in DMS and the upstream structure of the striatum are also involved in the formation of abrupt change of learning.However,mice with pathological above-mentioned structure or function will show deficits in the negative reinforcement learning.Therefore,reinforcement learning performance can be considered as a method to assess autism.