A Reinforcement Learning-based Method For Optimizing The Sequence Of Points In Acupuncture Treatment For Insomnia

For the treatment of insomnia,acupuncture treatments in Traditional Chinese Medicine(TCM)have the advantages of fewer side effects,quicker results,and lower prices compared to drugs and psychological and physiological treatments.It has been proven that acupuncture treatments at acupuncture points such as Baihui,Sanyinjiao,and Zusanli are effective in treating insomnia.Clinicians often select multiple acupuncture points for acupuncture stimulation in a single acupuncture treatment.However,because the specific effects and priority order of individual acupuncture points on disease are usually only recorded in ancient acupuncture literature,there needs to be more exploration of the impact of the order of acupuncture operations performed at multiple acupuncture points on the priority of therapeutic effects.At the same time,in current clinical practice,the determination of the order of acupuncture operations to be performed at multiple acupuncture points inpatient treatment relies mainly on the treatment experience of the practitioner,and there is no transparent quantitative model or evaluation method.Therefore,it is essential to explore the optimization of the sequence of multiple acupuncture points in treating insomnia with TCM acupuncture,both for the recovery of the insomnia patients themselves and for the efficient use of national health care resources.In addition,the primary tool currently for the diagnosis of insomnia,epilepsy,and other psychiatric disorders always relies on the observation of the patient’s EEG signals.At the same time,evaluating the effectiveness of rehabilitating the patients mentioned above requires a combination of the patient’s competent feelings and the physician’s evaluation of the patient’s EEG signal characteristics.Unfortunately,there needs to be more quantitative evaluation models for the efficacy of patients after acupuncture treatment.To address these problems,this paper proposes a reinforcement learning-based method for optimizing the order of acupuncture points for insomnia treatment.The following three aspects are the main contributions of this paper.1.Optimizing the order of acupuncture points for multiple acupuncture point treatment is essentially a sequential decision problem.In this paper,a reinforcement learning approach is applied to TCM acupuncture treatment,and a reinforcement learning-based method for optimizing the order of acupuncture point treatment for insomnia treatment is proposed.Its design steps and procedures are described in detail.Among them,this paper gives a neural network-based prediction model of EEG signals after acupuncture treatment to address the state transfer required for reinforcement learning to interact with the acupuncture treatment environment.This prediction model assign the pre-acupuncture treatment EEG signals as input and the postacupuncture treatment EEG signals as output to represent the state transfer of the acupuncture treatment environment,respectively.2.To address the problem of a lack of quantitative models of acupuncture treatment efficacy,this paper proposes a quantitative model of acupuncture treatment efficacy based on EEG signals.The aim is to construct a real-time quantitative model for efficacy evaluation based on the EEG signal characteristics of patients to get rid of the subjective experience of physicians.Subsequently,to verify the correctness and validity of the proposed efficacy quantification model,we applied the model to the clinical epilepsy and insomnia data such as,and obtained satisfactory results.3.Finally,to address the problem of optimizing the sequence of acupuncture points for insomnia treatment,we collaborated with two acupuncturists with associate or higher titles at the Institute of Acupuncture and Moxibustion,Chinese Academy of Traditional Chinese Medicine.We recruited thirty patients with insomnia to acquire EEG signals during acupuncture treatment for insomnia.The collected EEG data were used to quantify the efficacy of acupuncture and model the post-acupuncture treatment EEG signal prediction.They were applied to the reinforcement learning-based optimization of acupuncture treatment point sequences,obtaining better results and verifying the effectiveness and feasibility of the method proposed in this paper.