Stochastic Optimal Control Problem And The Weak Solution Research Of The Related Hamilton-Jacobi-Bellman Equation

Dynamic programming, originated by R. Bellman [10] in the early 1950s, is amathematical technique for making a sequence of interrelated decisions, which can beapplied to many optimization problems (including optimal control problems). J. Yongand X. Zhou in [92] discussed the stochastic optimal control problem and they gave thecelebrated dynamic programming and the viscosity solution of the corresponding HJBequation.General nonlinear BSDE was first introduced by Pardoux and Peng [69] in 1990.They showed that there is a unique adapted solution when the coefficient was Lipshctzcontinuous. Independently, Duffie and Epstein [27] introduced stochastic differentialutilities in economics, as solutions to a certain type of BSDEs. After that time, this kindof equations has received considerable research attention due to their nice structure andwide applicability in numbers of different areas, such as mathematical finance, stochasticcontrol, economical management and etc.. In 1992, Peng [74] got the Bellman's dynamicprogramming principle for a kind of stochastic optimal problem whose cost function wasdescribed by a BSDE and proved the value function is a viscosity solution of a generalHJB equation.Later in 1997, El Karoui, Kapoudjian, Pardoux, Peng and Quenez [46] generalizedthe equations to reflected case, that is. the solution is forced to stay above a givenstc chastic process, which is called the obstacle. They introduced an increasing processto push the solution upward. Furthermore, the push is minimal. The equation is in theform of Wu and Yu in [87] studied this stochastic recursive optimal control problem with theobstacle constraint for the cost functional, i.e. the cost functional of the control systemis described by the solution of a reflected BSDE with one lower barrier. They provethat the dynamic programming principle holds and the value function of this optimalcontrol problem is the viscosity solution of the corresponding HJB equation.However, there was a major drawback in the classical dynamic programming approach:It required that the HJB equation admit classical solutions, meaning that thesolutions be smooth enough (to the order of derivatives involved in the equation). Unfortunately,this is not necessarily the case even for some very simple situations. Inthe stochastic case where the diffusion is possibly degenerate, the HJB equation mayin general have no classical solution either. To overcome this difficulty, those works allgave the viscosity solution for the HJB equation. In this thesis, we first attempted todiscuss the Sobolev solution for the HJB equation.The other fact in most of these works is that an optimal solution in the class ofstrict controls may fail to exists. Existence of such a strict optimal control follows fromthe Filippov convexity condition. Without this convexity condition, an optimal strictcontrol does not necessarily exist in U. To overcome this problem of existence withoutimposing the Filippov condition, we discuss the relaxed control in this thesis. In thefollowing, we list the main result of this thesis.Chapter 1: We introduce problems studied from Chapter 2 to Chapter 4.Chapter 2: We study the dynamic programming principle for the stochastic relaxedoptimal control problem and prove that the value function is the unique weaksolution in Sobolev space for the related HJB equation, we also study the case whenthe cost functional is recursive the value functional is also the Sobolev weak solution.Theorem 2.2.4. (Dynamic Programming Principle) Let (A2.3) (A2.4) hold, thenfor any (t,x) G [0, T)×RⁿTheorem 2.3.1. Suppose (A2.3)-(A2.4) hold and the value function V∈C^1,2([0,T]× Rⁿ). Then V is a solution of the following second-order partial differential equation:whereTheorem 2.3.8. (Sobolev Weak Solution of The HJB equation) Under the assumption(A2.5)-(A2.9), the value function V(t,x) defined in (2.7) is the unique Sobolevsolution of the PDE (2.36).Theorem 2.5.5. Under the assumption (A2.5) (A2.6) and (A2.11)- (A2.13), the valuefunction V(t,x) defined in (2.90) is the unique Sobolev solution of the PDE (2.92).Chapter 3: We study the dynamic programming principle for the stochastic recursiverelaxed optimal control problem with the obstacle constraint for the cost functionaldescribed by the solution of a reflected BSDE and showed that the value function is theunique Sobolev weak solution of the obstacle problem for the corresponding Hamilton-Jacobi-Bcllman equation.Theorem 3.2.8.(Dynamic Programming Principle) Under the assumption (A2.3)-(A3.4), the value function u(t,x) obeys the following dynamic programming principle:for each 0＜δ≤T-t,Theorem 3.3.6. (Sobolev Weak Solution of The HJB equation) Under the assumption(A2.5)-(A2.6) and (A3.5)-(A3.8), the value function V(t,x) defined in (3.8)is the unique Sobolev solution of the PDE (3.15). Chapter 4: We study the stochastic recursive zero-sum differential game andmixed differential game problem and get the existence results of a saddle-point. Themain tool is backward stochastic differential equations (BSDEs) and double barrier reflectedBSDEs. We also discuss the American game option pricing problem when loaninterest rate is higher than the deposit one as the motivation and application background.Theorem 4.1.1. (Y^*, Z^*) is the solution of the following BSDE,Then Y₀^* is the optimal payoff J(x₀, u^*, v^*), and the pair (u^*, v^*) is the saddle point forthis recursive game.Theorem 4.2.1. (Y^*, Z^*, K^*+, K^*-) is the solution of the following reflected BSDEssatisfying (?)0≤t≤T,L_t≤Y_t^*≤U_t and t(?)(Y_s^*-L_s)dK_s^*+=(?)(Y_s^*-U_s)dK_s^*-= 0,We defineÏ„^*=inf{s∈[0,T],Y_s^*=U_s} andθ^*=inf{s∈[0,T],Y_s^*=L_s}.Then Y₀^*=J(x₀;u^*,Ï„^*;v^*,θ^*),(u^*,Ï„^*;v^*,θ^*) is the saddle-point strategy.Example 4.3.1 American game option when loan interest is higher than deposit interest.We give one example with explicit optimal solution of the saddle-point is also givento illustrate the theoretical results.