Use of adjoint physics in 4D var with the NCEP global spectral model

Variational analysis experiments were carried out to find optimal initial conditions and parameters for the 1995 versions of the adiabatic and diabatic NCEP MRF global spectral weather forecasting models. Optimal values were found by minimizing a cost function defined as the time-integrated 6-hour squared forecast error. The minimization process is complicated since the cost function is not a convex function of the parameters but has discontinuous jumps due to model physics.; Past work has concentrated on smoothing parameterization in physics by replacing discontinuous IF statements with smooth transitional functions. Our investigation on influences of parameterized physics on variational analysis establishes five main findings: (1) The adjoint of a diabatic model containing parameterized physics can correctly evaluates the gradient of the cost function even when there are local discontinuities. (2) The quasi-Newton algorithm designed for differentiable functions can usually successfully find the stationary point of the cost function of adiabatic model as well as an adiabatic model. (3) The stationary point found by the minimization algorithm may not occur at the minimum of the cost function because the cost function of a diabatic model is piecewise differentiable, and sometimes the algorithm fails. (4) Introduction of a smooth transitional function into physical parameterizations may not help the minimization process but, under certain specific situations, the minimization process remains stuck around the "introduced" local stationary point thus failing to approach a real minimum. (5) A nonsmooth minimization algorithm (bundle) successfully finds the minimum of the discontinuous cost function when the quasi-Newton algorithm fails, but the bundle method involves a computational cost which is almost double that of the quasi-Newton method. Parameter estimation and data assimilation procedures are used to examine these conclusions.; Parameterized physics introduce both discontinuity and nonlinearity into a diabatic model. The nonlinearity slows down the rate of decrease of the cost function when the physics play an important role. After 50 iterations, the cost function of the diabatic model decreases by about 75% while the cost function of the adiabatic model decreases 84%. Statistical verification is carried out to reveal improvement of forecast skills by introducing the optimally estimated parameters including the Asselin filter coefficient, the horizontal diffusion coefficient and initial conditions. For the adiabatic model, RMS forecast errors are lower with the optimal values than with the "operational" values for forecasts out to 10 days and beyond. For the diabatic models, optimal estimated parameters reduce the RMS errors only out to 3 days. Beyond 3 days, forecast skills are similar. This phenomenon is caused by two factors: (i) imperfect models, which affect the optimality of the initial conditions for forecasts beyond the optimization interval, (ii) intrinsic loss of predictability with increasing forecast lead time, particularly at small-scales.