Forward And Backward Stochastic Optimal Control Theory With Poisson Jumps And Its Applications

Stochastic optimal control problem is important in modern control theory. Thiskind problem always asks the controllers to minimize/maximize some cost functionaland subject to a state equation (stochastic control system) over the admissible controlset. An admissible control is called optimal if it achieves the infimum/supremum of thecost functional and the corresponding state variable and the cost functional are calledthe optimal trajectory and the value function, respectively. It is well-known that Pontryagin's maximum principle and Bellman's dynamic programming are the two principaland most commonly used approaches in solving stochastic optimal control problems. Inthe statement of maximum principle, the necessary condition of optimality is given. Thiscondition is called the maximum condition which is always given by some Hamiltonianfunction. The Hamiltonian function is denned with respect to the system state variableand some adjoint variables. The equation that the adjoint variables satisfy is calledadjoint equation, which is one or two backward stochastic differential equations (BSDEsfor short) of Pardoux-Peng's type. The system which consists of the adjoint equation,the original state equation, and the maximum condition is referred to as a generalizedHamiltonian system. On the other hand, the basic idea of dynamic programming principle is to consider a family of stochastic optimal control problems with different initialtime and states and establish relationships among these problems via the so-called HJBequation, which is a nonlinear second-order partial differential equation (PDE for short).If the HJB equation is solvable, we can obtain an stochastic optimal control by takingthe maximizer/miminizer of the generalized Hamiltonian function involved in the HJBequation. This is called stochastic verification theorem (SVT for short). Both of thesetwo approaches have been developed separately and independently, and recently thereare some researches on the relationship between these two approaches in literatures. The main objective of this thesis is to improve and develop the stochastic optimalcontrol theory, especially for forward-backward problems with Poisson jumps. Stochastic differential equations with Poisson jumps (SDEPs for short), backward stochasticdifferential equations with Poisson jumps (BSDEPs for short) and forward-backwardstochastic differential equations with Poisson jumps (FBSDEPs for short) are usuallyinvolved in this kind of problem. The solutions to this kind equations are discontinuous,since the random disturbance in these equations comes from both Brownian motions andPoisson random measures. The Poisson random measure can be described as the counting measure associated with a jump process. More precisely, Poisson random measurecounts the number of jumps of some discontinuous process occurring between a timeinterval whose amplitude belongs to a measurable set. That is to say, the Poisson random measure contains all information about some discontinuous (jump) process: it tellsus when the jumps occur and how big they are. The forward and backward stochasticoptimal control theory with Poisson jumps has wide practical applications in engineeringand financial market.In Chapter 2, we investigate the relationship between maximum principle (MP forshort) and dynamic programming principle (DPP for short) for stochastic optimal controlproblem of jump diffusions. Here, the system state process is described by a controlledSDEP. Firstly, on some mild assumptions we give some basic properties of the valuefunction and prove that the DPP still holds in our jump diffusion setting. Then we givethe corresponding generalized HJB equation which is a second-order partial integraldifferential equation (PIDE for short) containing the generalized Hamiltonian functionnow. Secondly, on the assumption that the value function is smooth, we establish therelationship between stochastic MP and DPP. Thirdly, using the theory of viscositysolutions, we also obtain the relationship between stochastic MP and DPP withoutassuming the smoothness of the value function. Finally, an SVT is at first derivedon the assumption that the value function is smooth from which we can obtain theoptimal control by maximizing the generalized Hamiltonian function. Then anotherversion of SVT is proved without involving any derivatives of the value function withinthe framework of viscosity solutions.Nonlinear BSDE was introduced by Pardoux and Peng[74] firstly. Independently,Duffie and Epstein[35] also introduced BSDE from economic background. In [35], theypresented a stochastic differential formulation of recursive utility. Recursive utility is anextension of the standard additive utility with the instantaneous utility depending not only on the instantaneous consumption rate but also on the future utility. As found by ElKaroui, Peng and Quenez[37], the utility process can be regarded as a solution to a specialBSDE. From BSDE's point of view, [37] also gave formulations of recursive utility andtheir properties. Thus, the problem whose cost functional of a control system is describedby the solution to a BSDE, becomes a stochastic recursive optimal control problem. InChapter 3, we consider one kind of stochastic recursive optimal control problem withPoisson jumps, where the cost functional of the control system is described by the solutionto a BSDEP. For this problem, using the notion of stochastic backward groups introducedin Peng[79], Li and Peng[59] recently have obtained the corresponding DPP and provedthat the value function is a viscosity solution to some generalized HJB equation. We theninvestigate the relationship between MP and DPP for such problem. For this purpose, wefirstly prove a local MP for the forward-backward stochastic control system with Poissonjumps. Moreover, we prove that under some additional convexity/concavity conditions,the above MP is also sufficient. Applications of our result to a mean-variance portfolioselection mixed with a recursive utility functional optimization problem in the financialmarket is discussed. Then, on the assumption that the value function is enough smooth,we obtain the relationship between stochastic MP and DPP. As applications, we givean example of linear quadratic (LQ for short) recursive portfolio optimization problemin the financial market. In this example, the optimal control in state feedback form isobtained by both the stochastic MP and DPP, and the relations we obtained are verified.The LQ stochastic optimal control problems are the most important examples ofstochastic optimal control problems, especially due to their nice structures and wideapplications in engineering design. In Chapter 4, we study one kind of coupled forward-backward LQ stochastic optimal control problem with Poisson jumps. Such kind ofoptimal control problems can be encountered in the financial market when we considerthe "large investor". We prove that there exists a unique optimal control and give theexplicit linear state feedback form. When all the coefficient matrices are deterministicwe can give the linear state feedback regulator for the optimal control using the solutionto one kind of generalized matrix-valued Riccati equation system. And the solvability ofthis kind Riccati equation system is discussed.SDE whose coefficients are modulated by a continuous-time Markov chain stemsfrom the regime switching models in financial market for the need of more realistic models that better reflect random market environment. In the regime switching model, themarket parameters depend on the market mode that switches among a finite number of states. The market mode could reflect the state of the underlying marketplace and thegeneral mood of investors, and other economic factors. In her doctoral thesis, Tang[97]recently introduced BSDEs with Markov chains whose generator are disturbed by random environment and described by a continuous-time Markov chains. Motivated byan LQ stochastic optimal control problem for Markov-modulated control system withPoisson jumps, in Chapter 5, we generalized part results of Tang[97] to discontinuouscase. That is to say, we consider BSDEPs with Markov chains. On the assumptionthat the generator satisfies the global Lipschitz condition, we obtain the existence anduniqueness of solutions to them, by virtue of some extended martingale representationtheorems. Some properties of these solution processes and a comparison theorem in theone-dimensional case are obtained.Another objective of this thesis is to study the partially observed fully coupledforward-backward stochastic optimal control problems. One of the most important characteristics for partially observed optimal control problems is that it has more practicalbackground. Specifically, controllers can not fully observe the system states in reality.In most cases they can only observe some noise process related to the system. Recently,more researching attentions have been attracted by optimal control problems of fully coupled forward-backward stochastic systems. One reason is that the theory is interestingand challenging in itself. Another is that these kinds of systems are usually encounteredwhen we study some financial optimization problems for some "large investors". In thiscase, state processes are described as fully coupled FBSDEs. In Chapter 6, on the assumption that the the control domain is possibly not convex, we obtain a stochasticMP for one kind of partially observed fully coupled forward-backward stochastic optimalcontrol problem by spake variational, duality and filtering techniques. To illustrate ourtheoretical result, we give an example for a partially observed fully coupled LQ forwardbackward stochastic optimal control problem. Combining the classical linear filteringtheory with the technique of solving linear FBSDEs, we find an explicit observable optimal control. Meanwhile, we obtain the filtering estimates of the optimal trajectorieswhich are given by the solutions to some forward-backward ordinary differential equationwith double dimensions (DFBODE for short) and Riccati equations. Finally, problemwith state constraints is discussed by combining Ekeland's variational principle with thetechnique presented above.This thesis consists of six chapters. In the following, we list the main results.Chapter 1: We introduce problems studied from Chapter 2 to Chapter 6. Chapter 2: We establish the relationship between maximum principle and dynamicprogramming principle for the stochastic optimal control problem of jump diffusions. Weconsider the following stochastic control systemand the cost functional isThe stochastic optimal control problem of jump diffusions is the following.Problem (JD)_s,y. For given (s,y)∈[0, T)×Rⁿ, minimize (2.2) subject to (2.1) overU[s,T].The main results are the following Theorem 2.4 for smooth value function andTheorem 2.8 for nonsmooth value function.Theorem 2.4. (Relationship, Smooth Case) Suppose (H2.1)～(H2.3) hold and let(s,y)∈[0,T)×Rⁿ be fixed. Let (?) be an optimal pair for Problem (JD)_s,yand (?) the solution to the first-order adjoint equation (2.19). Suppose thevalue function V∈(?). Thenwhere G is defined by (2.16). Further, if V∈(?) and V_tx is also continuous,thenTheorem 2.8. (Relationship, Nonsmooth Case) Suppose (H2.1)～(H2.3) holdand let (s, y)∈[0,T)×Rⁿ be fixed. Let V∈(?), satisfying (2.8) and (2.9), be a viscosity solution to the generalized HJB equation (2.15), (?) be the optimalpair for Problem (JD)_s,y. Let (?) and (?) are solutions tothe first- and second-order adjoint equations (2.19), (2.20), respectively. Then we havewhere G-function is defined by (2.54).Moreover, the following two results give the stochastic verification theorems forsmooth and nonsmooth value function, respectively.Theorem 2.9. (SVT, Smooth Case) Suppose (H2.1)～(H2.3) hold. Let V∈(?) be a solution to the generalized HJB equation (2.15). ThenMoreover, suppose a given admissible pair (?) satisfieswhere G is defined by (2.16). Then (?) is an optimal pair.Theorem 2.10. (SVT, Nonsmooth Case) Suppose (H2.1), (H2.2) hold. Let V∈(?), satisfying (2.8) and (2.9), be a viscosity solution to the generalized HJBequation (2.15). Then we have the following.(i) (2.73) holds.(ii) Let (s,y)∈[0,T)×Rⁿ be fixed and let (?) be an admissible pair.Suppose there exists (?),such thatand where (?), such that (?).Then (?) is an optimal pair.Chapter 3: We establish the relationship between maximum principle and dynamicprogramming principle for the stochastic recursive optimal control problem with Poissonjumps. As preliminaries, we firstly consider the following forward-backward stochasticcontrol systemand the cost functional isThe forward-backward stochastic optimal control problem is the following.Problem (FB)_0,T. For given x₀∈Rⁿ, minimize (3.2) subject to (3.1) over U_ad.Using convex variational method, we first prove a local maximum principle.Theorem 3.1. (Local Stochastic Maximum Principle) Suppose (H2.1), (H2.3)',(H3.1) and (H3.2) hold. Let u(·) be an optimal control for Problem (FB)_0,T, and(?) be the corresponding optimal trajectory. Then we havewhere the Hamiltonian function H is defined by (3.7).Moreover, under some additional convexity/concavity conditions, the above necessary condition in Theorem 3.1 is also sufficient.Theorem 3.2. (Sufficient Condition for Optimality) Suppose (H2.1), (H2.3)',(H3.1)～(H3.3) hold. Let u(·) be an admissible control and (?) be thecorresponding trajectory with y(T) = (?). Let (?)be the solution to adjoint equation (3.6). Suppose that H is convex with respect to (?). Then u(·) is an optimal control for Problem (FB)_0,T if it satisfies(3.8).Then we investigate the relationship between maximum principle and dynamic programming principle for stochastic recursive optimal control problem with Poisson jumps.We consider the following stochastic control systemand the cost functional isThe stochastic recursive optimal control problem is the following.Problem (R)_s,y. For given (s,y)∈[0,T)×Rⁿ, minimize (3.31) subject to (3.35) overU[s,T].The main result is the following.Theorem 3.6. (Relationship, Recursive Problem, Smooth Case) Suppose (H2.1),(H2.3)', (H3.1), (H3.2) hold and (?) be fixed. Let u(·) be an optimal control for Problem (R)_s,y, and (?) be the corresponding optimal trajectories. Let (?) be the solutions to adjointequation (3.36). Suppose that the value function (?). Then Further, if (?) and V_tx is also continuous, thenChapter 4: We study one kind of coupled forward-backward LQ stochastic optimalcontrol problem with Poisson jumps. We consider the following stochastic control systemand the cost functional isThe LQ stochastic optimal control problem is the following.Problem (LQ)_0,T. For given x₀∈Rⁿ, minimize (4.6) subject to (4.5) over U_ad.We prove that there exists a unique optimal control and give the explicit linear statefeedback form.Theorem 4.1. There exists a unique optimal control for Problem (LQ)_0,T: where (?) is the corresponding optimal trajectory.When all the coefficient matrices are deterministic, we can give the linear statefeedback regulator for the optimal control using the solution to one kind of generalizedmatrix-valued Riccati equation system.Theorem 4.2. Suppose for all t∈[0,T], there exist matrices (?)satisfying the generalized matrix-valued Riccati equation system (4.9). Then the optimallinear state feedback regulator for Problem (LQ)_0,T isand the optimal value function isThe solvability of this kind of generalized matrix-valued Riccati equation system isdiscussed. In some special case, we obtain the following existence and uniqueness result.Theorem 4.5. Suppose (H4.3) holds and D≡0. Then the generalized matrix-valuedRiccati equation system (4.9) has a unique solution (?).Chapter 5: We study BSDEPs with Markov chains. As motivation, firstly wediscuss an LQ stochastic optimal control problem with Poisson jumps and Markov chains.We consider the following stochastic control systemand the cost functional iswhere (?) is a continuous-time Markov chain with the state space(?).αhas the transition probabilities given by where q_ij≥0 for i≠j and q_ij = (?).The LQ stochastic optimal control problem with Markov chains is the following.Problem (LQMC)_0,T. For given x₀∈Rⁿ, minimize (5.2) subject to (5.1) over U_ad.An optimal state feedback control and the value function is obtained via the solutionto a constrained stochastic Riccati equation.Theorem 5.1. If the constrained stochastic Riccati equation (5.4) admits a solutiontriple (?), then Problem(LQMC)_0,T is well-posed. Moreover, the optimal state feedback control is (with the timeargument t suppressed)Furthermore, the value function isMotivated by this kind of stochastic Riccati equations, we study the following BS-DEP with Markov chains:On the assumption that the generator satisfies the global Lipschitz condition, weobtain the existence and uniqueness result of the solution, by virtue of some extendedmartingale representation theorems.Theorem 5.2. (Existence and Uniqueness of solution to BSDEP with MarkovChains) Suppose (H5.1) holds. Then BSDEP (5.8) admits a unique solution tripleSome properties of these solution processes are obtained and a comparison theoremin the one-dimensional case is proved. For this target, let (?) beanother continuous-time Markov chain with the state space (?) has the transition probabilities given bywhere q_jk≥0 for j≠k and (?).Theorem 5.4. (Comparison Theorem) Suppose n =1. Let (?)satisfy (H5.2). Furthermore, let process (?) bemeasurable and satisfy 0≤l(t, e)≤C(1∧|e|), e∈E. We setfor all (ω, t, y, z,φ,i)(?).Let (?) and f' be defined asfor all (?), where (?)satisfies (H5.2).We denote by (Y,Z,K(·)) (respectively, (Y',Z',K'(·))) the unique solution triple toBSDEP (5.8) with the data (ξ,f) (respectively, (ξ',f')). Then, if(iv)ξ≥ξ', a.s.;(v) for Markov chains (?);(vi) f(t,y,z,k(·),i) is nondecreasing with respect to i∈Μand (?), a.s.,a.e., (?), it follows thatAnd if, in addition, we also assume that P(ξ>ξ') > 0, then P(Y(t) > Y'(t)) > 0, (?)t∈[0,T]. In particular, Y(0) > Y'(0).Chapter 6: We study one kind of partially observed fully coupled forward-backwardstochastic optimal control problem. We consider the following stochastic control systemwith the observation and the cost functional isThe partially observed stochastic optimal control problem is the following.Problem (PO)_0,T. For given x₀∈Rⁿ, minimize (6.7) subject to (6.4) and (6.5) overU_ad.Our main result is the following.Theorem 6.1. (Partially Observed Stochastic Maximum Principle) Suppose(H6.1)～(H6.3) hold. Let u(·) be an partially observed optimal control for Problem(PO)_0,T, (?) be the optimal trajectory and Z(·) be the corresponding solutionto (6.6). Let (P(·),Q(?)) be the solution to auxiliary BSDE (6.34) and (p(·),q(·),k(·))be the solution to adjoint FBSDE (6.35). Then we havewhere the Hamiltonian function H is defined by (6.36).To illustrate our theoretical result, we give an example for a partially observedfully coupled LQ forward-backward stochastic optimal control problem. We consider thefollowing stochastic control systemand observationThe cost functional isThe partially observed LQ stochastic optimal control problem is the following.Problem (POLQ)_0,T. For given x₀∈Rⁿ, minimize (6.40) subject to (6.38) and (6.39)over U_ad.Combining the classical linear filtering theory with the technique of solving linear FBSDEs, we find an explicit observable optimal control. In addition, we obtain the filtering estimates of the optimal trajectories which are given by the solutions toa forward-backward ordinary differential equation with double dimensions (DFBODE)and Riccati equations.Theorem 6.2. (LQ case, Observable Optimal Control and Filtering Estimatesof Optimal Trajectories) For Problem (POLQ)_0,T, an observable optimal controlu(·) is given by (6.47), where (?) are solutions to DFBODE (6.53) andΠ(·) is thesolution to Riccati equation (6.44), where (?) are given by (6.51).Moreover, the filtering estimates of optimal trajectories (?) are given byDFBODE (6.53) and (6.57), respectively, whereΣ(·) is the solution to Riccati equation(6.55).Finally, we consider problem with state constraints. We introduce the followingstate constraints.The partially observed stochastic optimal control problem with state constraints isthe following.Problem (POC)_0,T. For given x₀∈Rⁿ, minimize (6.7) subject to (6.4) and (6.5)under constraints (6.57) over U_ad.The main result is the following.Theorem 6.3. (Partially Observed Stochastic Maximum Principle with StateConstraints) Suppose (H6.1)～(H6.4) hold. Let u(·) be an partially observed optimalcontrol for Problem (POC)_0,T, (?) the optimal trajectory and Z(·) thecorresponding solution to (6.6). Then there exists a nonzero triple (?)with (?), and (?) which are solutionsto the adjoint BSDE (6.60) and FBSDE (6.61), respectively, such that the followingmaximum condition holds:where the Hamiltonian function H is defined by (6.59).