Diagnostic Prediction Method And Application Based On Multimodal Sensing Data

With the development of intellectualization and informationization technology,the construction of informationization intellectualization has sprung up in all walks of life in our country.Especially in the fields of industrial manufacturing and medical treatment,a large number of information data from different sources and forms can be generated every day.The information from each source and form is a kind of mode.A multimodal data sample can contain a variety of information labels,which are multimodal numbers.It is of great research value for the development of production in its field.In this thesis,a prediction and diagnosis model for multimodal sensing data is proposed.Based on multi-domain multimodal sensing data and the characteristics of time-series and non-time-series multimodal sensing data,corresponding diagnosis and prediction methods are proposed,and meaningful prediction results are obtained.Multimodal data can be divided into temporal data and non-temporal data.The multimodal data collected on the production assembly line are mostly non-sequential data,and each test item of each product corresponds to only one data.The physiological data collected by the hospital intensive care unit are time series data.Each physiological signal has been detected and recorded for a long time,and has many data points.Multimodal sensing data,especially the non-sequential multimodal sensing data produced in industrial production process,generally have the characteristics of many kinds of attributes,complex information storage structure and large amount of data redundancy.In this paper,a prediction and diagnosis model for multimodal sensing data is proposed based on time series and non-time series multimodal sensing data,and corresponding diagnosis and prediction are made for multimodal sensing data in different fields.In this paper,the multimodal sensing data of physiological signals in hospital intensive care unit and the multimodal sensing process data of automobile engine assembly line are analyzed and processed respectively,and the disease of patients and the performance and stability of engine speed are predicted respectively.The work of this paper is mainly divided into three aspects:(1)In this paper,according to the data characteristics of multimodal sensing data,the corresponding preprocessing methods are proposed.Aiming at the difference of time dependence between time-series multimodal sensing data and non-time-series sensing data,two corresponding pattern feature extraction methods are elaborated.(2)Aiming at the problem of prediction and diagnosis of time-series multimodal sensing data,according to the time-dependent relationship between data modes,a probability map model combining quadratic mode extraction with Allen interval relationship and potential variables based on Chinese restaurant process is constructed.The diagnosis and prediction of patients' diseases are realized by using time-series multimodal physiological signal data of patients.(3)For the diagnosis and prediction of non-sequential multimodal sensor data,aiming at the classification and prediction problems,this thesis proposes the dynamic optimal combination mechanism of regression algorithm and the regional weighted integrated classification method respectively.Combining with sparse self-encoder,the multimodal process of engine assembly line is used to detect sensor data and process sensing data.The following diagnosis and prediction are realized respectively:1)Diagnosis prediction of engine speed performance qualification in three working conditions: start-up stage,high-speed stage and idle stage.2)Diagnosis and prediction of mean and standard deviation of engine speed in three complete working conditions: start-up stage,high-speed stage and idle stage.