New Differenced Methods And Their Applications

In econometrics and statistics, nonparametric theories have be rigorously investi-gated, and nonparametric techniques have a bright prospect of applications. For applied economists and statists, the usage of these techniques is limited:there is a gap between the elaborate theories and practical applications, i.e., the optimal choices of the kernel order and bandwidth for practical applications; meanwhile it requires substantial compu-tational and programming experience. Differenced method is a convenient nonparametric technique, and provide an alternative for practitioners. At present, differenced methods in nonparametric model have the following questions:(1) For lack of accurate results in finite samples, the estimation effectiveness is not guaranteed in applications;(2) There exists an open problem-which difference sequence to be used in finite samples;(3) There is a confusion in methodology-how to understand the traditional differenced methods;(4) In the presence of contaminated data, less robust differenced methods can be used.In this dissertation, we summarize some differenced methods in nonparametric re-gression, propose new differenced methods, give a connection between elaborate theories and practical applications, and solve the above four questions. The dissertation is divid-ed into six chapters. In chapter one, we introduce the traditional differenced methods and existing problems. In chapters two, three, and four, by combining the differenced methods and least squares regression, we propose two estimation methods of variance and one estimation method of derivatives, which solve the questions (1), (2), and (3). In chapter five, by combining the differenced methods and least absolute deviation regres-sion, we propose a robust derivative estimation method, which solves the question (4). In chapter six, we discuss the advantages of differenced methods in applications and future development directions. Note that:in equidistant design, we present accurate analysis method for finite samples which solves question (1); The analysis method provides a benchmark for non-equidistant and random designs.Chapter two studies the estimation of error variance in nonparametric regression models. The traditional method is residual-based:estimate the regression function us-ing nonparametric techniques, and then estimate the variance by the sum of residual squares. The existing problems are as follows:the optimal choice of bandwidth based on the tradeoff between the estimation bias and variance, and the larger estimation variance at boundaries. To solve these problems, we present a definition-based estimation method. It is an error-estimated method:firstly, construct a sequence of symmetric second-order difference and estimate the error as the constant term using the least squares regression; secondly, construct a variance estimator based on estimated errors point-by-point, ig-noring the boundary points. Under three different smoothness assumptions, we obtain three variance estimators. All three estimators achieve the optimal mean square error-most differenced estimators do not achieve. To determine which estimator to be used in practice, we propose a rule of thumb by analysis of the mean square error which solves question (2). In addition, we demonstrate that all differenced estimators are equivalent to kernel estimators which solves question (3). The results of this chapter have been published in Computational Statistics and Data Analysis. Chapter three presents a new method of variance estimation called the parameter-ized method:construct a sequence of variance estimators by the lagged differences, and then estimate variance as the constant term using the least squares regression. The vari-ance estimator achieves the optimal rate of the mean square error, meanwhile reduces the estimation bias. In addition, a correction method is proposed in non-equidistant design. The results of this chapter have been published in Computational Statistics. Chapter four studies the derivative estimation in nonparametric regression model-s. Usually people focus on the estimation of regression functions, the nonparametric derivatives estimation has never attached much attention since the derivatives often are estimated indirectly as'by-products'. Recently, the applications of derivative estimation are wide-ranging, which need new estimation methods. Charnigo et al. [2011b] propose a new estimate-empirical derivatives, by combining linearly the symmetric differences to reduce estimation variance. De Brabanter et al. [2013] further study the empirical derivatives and establish its consistency. However, the empirical derivatives have the following problems:the larger estimation bias in valley and spike and the wrong crite-rion based on regression function estimation but not on derivative estimation. To solve these problems, we proposed a new method of derivative estimation-locally weighted least squares regression:firstly, construct a sequence of symmetric difference quotients; secondly, obtain a sequence of regression expressions by applying the Taylor expansion to the difference quotients; thirdly, estimate the derivative as the constant term by the locally weighted least squares regression. The method reveals the disadvantage of local polynomial regression- the aim of local polynomial regression is to estimate the regres-sion function, thus the criterion focuses on the estimation of regression function, not of derivative function; the convergence rates are different between the mean estimator and derivatives estimators; the mean estimator achieves the optimal rate while the deriva-tives do not. We use the differenced method so that the interested derivative has the fastest rate of convergence. In addition, the method gives a regression interpretation on empirical derivatives, and reduces the bias in the valleys and peaks of the true derivative function-solve the problem (1). The results of this chapter have been published in Journal of Machine Learning Research.Chapter five combines the differenced methods and least absolute deviation regres-sion to propose a robust derivative estimation method-locally weighted least absolute deviation regression. Recall that the original least absolute deviation regression depends on the density value at the median, meanwhile needs the sharp density to keep the esti-mation efficiency. The new estimator has the more robust property:it does not depend on the density value at the median, but the expectation of the whole density function; even for the platykurtosis density, the differnced error becomes the sharp density. It is worth mentioning that our estimator has the similar property to composite quantile estimator, since they both rely on the expectation of the original density. The composite quantile estimator contains all information of the original density by using all regression quantiles, so it is more effective than our estimator, even its weighted version achieves the optimal convergence rate. In practical applications, composite quantile estimator only employs finite regression quantiles which equals to rely on finite density values; so that its efficiency surely reduces and is no guarantted. While our estimation efficiency is consistent to the theoretical resule.This work introduces the differenced methods systematically in nonparametric re-gression, reveals the close connection between differenced method and kernel method, proposes some new differenced methods, and solves some problems in theories and ap-plications. The future works are as follows in chapter six. Firstly, discuss the optimal theory and finite sample results for robust methods; Secondly, apply the differenced methods to different models:parametric model, semi-parametric model, multiple regres-sion model, and high-dimensional model; Thirdly, testing based on differenced methods is a new and virgin land, and needs seminal works; Last but not the least, apply the differenced methods to real data, i.e., satellite data, automatic data, engineering data, and econometric data.