Time-inconsistent Stochastic Control Problems And Forward-backward Stochastic Differential Equations In Weak Formulation

In this thesis, we mainly study two kinds of time-inconsistent stochastic control problems which do not admit Bellman’s optimality principle:one is a time-inconsistent optimal control problem with random coefficients, and the other is a partially observed time-inconsistent recursive optimization problem. We also investigate the maximum principle for a kind of recursive optimal control problem with obstacle constraint, whose cost functional is given by a reflected backward stochastic differential equation (BSDE). Moreover, Inspired by the close connections with stochastic optimal control theory as well as other applications, we introduce a new kind of forward-backward stochastic differential equations (FBSDEs) in weak formulation and discuss the wellposedness of such weak FBSDEs.Let us give the main content and organization of this thesis.In Chapter 1, we make a brief introduction of the historical backgrounds, research motivations and theoretical tools of the following chapters.In Chapter 2, we study a kind of time-inconsistent optimal control problems with random coefficients. By the method of multi-person differential games, a family of pa-rameterized backward stochastic evolution equations, called the stochastic equilibrium Hamilton-Jacobi-Bellman (HJB) equation, is derived for the equilibrium value function of this problem. Under appropriate conditions, we obtain the wellposedness of such an equation and construct the time-consistent equilibrium strategy of closed-loop. Besides, we investigate the linear-quadratic problem as a special and important case.In Chapter 3, we study a partially observed time-inconsistent recursive optimization problem. We first obtain the verification theorem for equilibrium control and a Hamil- tonian system which is a family of parameterized FBSDEs for the corresponding fully observed time-inconsistent recursive optimization problem, and establish the Kalman-Bucy filtering equations for such a Hamiltonian system. By means of the backward separation theorem, the equilibrium control for the partially observed time-inconsistent recursive optimization problem is obtained, which is a feedback of the filtering state trajectory. To illustrate the applications of theoretical results, an partially observed in-surance premium policy problem is presented and the observable equilibrium policy is derived explicitly.In Chapter 4, we study a kind of recursive optimal control problem whose cost functional is described by the solution of a reflected BSDE. Through approximating the reflected BSDE by a family of ordinary BSDEs via penalization, we establish the maximum principle of near-optimality for such a problem. We also obtain the sufficient conditions of optimal and near-optimal controls respectively. Besides, a mixed optimal control problem is considered to illustrate the application of our theoretical results and the optimal control and stopping strategy is given.In Chapter 5, we introduce a new kind of FBSDEs in weak formulation. By review-ing the application in option hedging theory, nonlinear Feynman-Kac formula and the relationship between maximum principle and dynamic programming principle, we claim that this kind of weak FBSDEs seems more natural and suitable. Especially, we present two examples to show that the weak FBSDEs associate with the stochastic optimal con-trol problems in weak formulation, which admit the optimality under milder conditions than those in strong formulation. Moreover, we investigate the wellposedness of such weak FBSDEs.In the following, we give the main results of this thesis.1. Time-Inconsistent Optimal Control Problem with Random Coefficients and Stochastic Equilibrium HJB Equation.For a given completed probability space (Ω,F, P) and two independent 1-dimensional and d-dimensional standard Brownian motions {Wt,t≥0},{Wt1,t≥0} on it, consider the following control system: and the cost functional: where b(x,v,s):Rn×Rk×[0,T]â†'Rn,σ(x,s):Rn ×[0,T]â†'Rn,Ï€(x,s):Rn×[0,T]â†' Rn×d,L(x,v,s,t):Rn×Rk×[0,T]×[0,T]â†' R are the deterministic functions, and h(x,t,ω):Rn×[0,T]×Ωâ†'R is a FTW-measurable random variable. An admissible control v on the time interval [t,T] is a FtW,W1-adapted process with values in U(?)Rk, such that E[∫tT|vs|2ds]<+∞. We call this time-inconsistent control problem with random coefficients Problem (N).Through defining the crucial mapping φ in (2.7), constructing a sequence of multi-person differential games indexed by the partition II and then investigating the situation when the mesh size||Π||â†'0, we establish the following stochastic equilibrium HJB equation, which is a family of parameterized backward stochastic evolution equations: where H=L2(Rn), V={u∈L2(Rn):Du∈L2(Rn)} and By the method of contraction mapping, we have:Theorem 2.3.1. Let Assumption 2.3.1 and 2.3.2 hold. Then there exist unique adapt-ed pairs (Θ.(·;Ï„),A.(·;Ï„)) ∈M2(Ï„,T;V)×M2{Ï„:T;H),0≤t≤T, satisfying the stochastic equilibrium HJB equation (36).Thus, we obtain the time-consistent equilibrium control and equilibrium value func-tion for Problem (N) in the sense of Definition 2.2.1:Theorem 2.3.2. Let Assumption 2.3.1,2.3.2 and 2.3.3 hold. Then Θt(x;t) is an equilibrium value function of Problem (N) for any initial data {x,t)∈Rn×[0,T], which is the solution to stochastic equilibrium HJB equation (36). Moreover, the corresponding time-consistent equilibrium control is given by (2.29).Besides, we study a time-inconsistent linear-quadratic (LQ) control problem, whose dynamic system becomes: and the cost functional is where for any s∈[0,T], A(·), C(·), G(·)∈Rn×n, S(·), F(·),H(·)∈Rn×k, Q(·) is a FTW-measurable nonnegative bounded random variable with values in Sn; for any (s, t)∈ D[0,T], R(·,·) ∈Sn is nonnegative, N(·,·)∈ Sk is positive.In the case that F=0, we take advantage of the quadratic structure to derive the. Riccati-Volterra integral equation system for this time-inconsistent LQ problem: As for the existence and uniqueness of solutions to (37), we have:Proposition 2.4.1. Suppose and Then (37) admits a unique solution.Therefore,for this time-inconsistent linear-quadratic control problem,we get the following result:Theorem 2.4.1. Let all assumptions for Proposition 2.4.1 hold.Then the equilibrium value function for the initial data(x,t)∈Rn×[0,T]is where K(.)is the solution of(37).Moreover,the time-consistent equilibrium control is given by(2.39).2. Partially Observed Time-Inconsistent Recursive Optimization Problem and Application.For a given completed probability space(Ω,F,P),a 2-dimensional standard Brow-nian motion{(W1(t),W2(t)),t≥0} and an independent Gaussian random variable ζ on it,we first consider a fully observed time-inconsistent recursive optimization problem, whose evolution is and the cost functional is where the coefficients A(·),B(·),C1(·),C2(·),a(·),b(·),c(·),f1(·),f2(·)are all Ftζ,W1,W2-adapted processes with values in R and g,h,μ1,μ2 are all constants. An admissible control u(·)is a Ftζ,W1,W2-adapted process with values in R,such that E[∫0T|u(t)|4dt]< +∞.We denote the set of al admissible controls by U.Redefine a probability measure Q on the space(Ω,F)by According to Girsanov’s Theorem,the process{(U(t),V(t)),t≥0)given by is a 2-dimensional Q-standard Brownian motion. Thuw,by standard computation,the cost functional(39)can be rewritten as: where EtQ[·]=EQ[·|Ftξ,W1,W2]denotes the conditional expectation with respect to Ftξ,W1,W2 on the space(Ω,F,Q).Inspired by the spike variation method of maximum principle,we obtain a sufficient condition of equilibrium control in the sense of Definition 3.1.1 for this time-inconsistent control problem: Theorem 3.1.1. Let Assumption 3.1.1 hold.If there exist processes{(X*(s),u*(s)),0≤ s≤T) and a family of processes{(p(sï¼›t),k1(s;t),k2(sï¼›t)),t≤s≤T),0≤t≤T such that u*∈U and for any t∈[0,T),they satisfy the following Hamiltonian system and A(·;t)=B(.)p(·;t)+2c(·)e∫tb(r)dru*(·)satisfies(3.8),then u* is an equilibrium control.However,in many situations,we cannot observe(X*,p,k1,k2)in(41)directly,but observe a noisy process Z(·)related to X*(·),whose dynamic system is To get the best estimation of(X*,p,k1,k2)with respect to the observation Z(·),denoted by(X*,p,k1,k2),we decouple the Hamiltonian system(41)and compute by classical filtering theory for forward SDEs to obtain the following result:Theorem 3.2.1. Let Assumption 3.1.1 and 3.2.1 hold.Then the best filtering esti-mation{(X*(s),p(s;t),k1(s;t),k2(s;t)),t≤s≤T}of solutions to Hamiltonian system (41),0≤t≤T, are given by (3.21),(3.22), (3.24) and (3.25), where M(·), N(·),Γ(·) and φ(·) are solutions of (3.16), (3.17), (3.18) and (3.19) respectively.Then we study the corresponding partially observed time-inconsistent recursive op-timization problem. By virtue of backward separation theorem, we separate the state and observation equations as follows: where an admissible control u(·) is a FlZ and FtZ1-adapted process with values in R, such that E[∫0T|u(t)|4dt]<+∞.Thus, combining with the results above, we can obtain the equilibrium control for the partially observed time-inconsistent recursive optimization problem as follows:Theorem 3.3.1. Let Assumption 3.1.1 and 3.2.1 hold. Then the equilibrium control for the partially observed time-inconsistent recursive optimization problem is given by (3.31), where M(·), N(·), Γ(·) and φ(·) are solutions of (3.16), (3.17), (3.18), (3.19) and X*(·) is the filtering state trajectory under the equilibrium control (3.31), whose evolution is described by (3.33).At last, we study an insurance premium policy problem as the application of our theoretical results. Consider an insurance firm whose cash-balance process X(·) is where x0> 0 is the initial capital, the risk-free interest rate δ(·)> 0, the liability rate l(·)> 0 represents the expected liability per unit time, the premium rate v(·) acts as the control variable and the volatility rate σ(·)>0 measures the liability risk.This firm wants to make an optimal premium rate v(·) to minimize the cost: where the constant β is a discounting factor, c0 is some pre-set target, G, Q and process R(·) are the weighting factors which make the cost (46) more general and flexible. How- ever, the policymaker cannot observe the cash-balance process X(·) directly, but observe the stock price S(·) which is closely related to X(·) as follows: where the constants a, c are correlation coefficients, and the process p(·) is the volatility rate.By some standard change in variables and computation, this control problem can be reformulated as a partially observed time-inconsistent recursive optimization problem. Thus we obtain the equilibrium premium policy in the following: Theorem 3.4.1. Let Assumption 3.4.1 and 3.4.2 hold. Then the observable equilibri-um premium policy is where J1(·) and φ1(·) are given by (3.58) and (3.59) explicitly, and X*(·) is the filtering cash-balance process under the equilibrium premium policy, whose evolution is described by (3.52).3. Stochastic Maximum Principle for A Kind of Recursive Optimal Control Problem with Obstacle Constraint.For a given completed probability space (Ω,F, P) and a d-dimensional standard Brownian motion {Wt, t≥0} on it, consider the following forward control system: and a controlled reflected BSDE: with the cost functional: where α∈Rd Rd is a given constant, and b(t,x,v):[0,T]×Rd×Rlâ†'Rd,σ(t,x) [0, T]×Rdâ†'Rd×d, f(t,x,y,v):[0, T]×Rd ×Rm×Rlâ†'Rm, h(t, x):[0, T]×Rdâ†'Rm, g(x):Rdâ†'Rm,γ(y):Rmâ†'R are all deterministic functions. An admissible control v is a FtW-adapted process with values in compact set U(?)Rl, such that E[∫0T|vt|2dt] +∞. Denote the set of all admissible controls by U. We call this recursive optimal control problem with obstacle constraint Problem (P).Suppose u ∈U is an optimal control of Problem (P), and {xt,0≤t≤T}, {(yt,zt,kt),0≤t≤T} are the corresponding solutions of (49) and (50), respective-ly. Since an extra increasing continuous process {kt} is introduced, we cannot apply the spike variation method directly to establish the maximum principle for Problem (P). At first, we construct a sequence of ordinary BSDEs via penalization to approximate the reflected BSDE (50): where n=1,2,…. Thus, according to the Ekeland’s variational principle, there exist a sequence of admissible controls {un}n≥1 and positive real numbers {εn}n≥1 decreasing to 0 given by (4.8), such that {un}n≥1 is near-optimal for Problem (P), while for each n∈N,un∈U and the corresponding solutions {xtn,0≤t≤T},{(ytn,ztn),0≤t≤T} of (49) and (52) are the optimality of the following optimal control problem: Problem (Pn) Minimize the cost functional over v∈U, subject to the forward-backward stochastic control system (49) and (52).However, the generator of (52) is just Lipschitz continuous but not differentiable, for which we still cannot apply the spike variation method directly. So, for any n, k∈N, define the smooth function: where φ,Ψ are two mollifiers. Now, we introduce BSDEs with smooth generators: where n,k=1,2,…. Similarly, according to the Ekeland’s variational principle, for any fixed n∈N, there exist a sequence of admissible controls {un,k}k≥1 and positive real numbers {δn,k}k≥1 decreasing to 0 given by (4.20), such that {un,k}k≥1 is near-optimal for Problem (Pn), while for each n,k ∈N, un,k ∈U and the corresponding solutions {xtn,k,0≤t≤T},{(ytn,k,ztn,k),0≤t≤T} of (49) and (55) are the optimality of the following optimal control problem: Problem (Pn,k) Minimize the cost functional over u∈U, subject to the forward-backward stochastic control system (49) and (55).Thus, by the standard spike variation method, we obtain:Proposition 4.2.1. Let Assumption 4.1.1,4.1.2 and 4.1.3 hold. Then for any fixed n∈N, there exists an admissible control un being the optimality of Problem (Pn) and a constant εn>0, and there also exist a sequence of admissible controls {un,k}k≥1 being near-optimal of Problem (Pn) and positive real numbers {δn,k}k≥1 decreasing to 0, such that for any k∈N, 1) d(un,k,un)≤(?); 2) For any v∈U, where{xtn,k},{(ytn,k,ztn,k)} are the solutions of (49) and (55) corresponding to the control un,k,{Ptn,k},{Qtn,k} are adjoint processes given by (4.22) and (4.23) respec-tively, and the Hamiltonian function Hn,k isThen, combining with the Krylov’s inequality, we investigate the situation when n∈N is fixed and k is sent to oo to establish the maximum principle for near-optimality of Problem (P):Theorem 4.2.1. Let Assumption 4.1.1,4.1.2,4.1.3 and Assumption 4.2.1,4.2.2 hold, and u∈U is an optimal control of Problem (P). Then there exist a sequence of admissible controls{un}n≥1 being near-optimal of Problem (P) and positive real numbers{εn}n≥1 decreasing to 0, such that for any n∈N, 1) d(un, u)≤(?); 2) For any v∈U, where{xtn},{(ytn,ztn)}are solutions of(49)and(52)corresponding to the control un,{Ptn),{Qtn}are adjoint processes given by(4.33)and (4.34) respectively,and the Hamiltonian function Hn is Moreover,by virtue of Clarke’s generalized gradient,we can also give the sufficientconditions for both optimal and near-optimal controls of Problem(P).Theorem 4.3.1. Let Assumption 4.1.1,4.1.2,4.1.3 and Assumption 4.3.1 hold.Sup-pose u is an admissible control and {xt,0≤t≤T},{(yt,zt,kt),0≤t≤T)are the corresponding solutions of(49)and(50),respectively.Denote Ï„*=inf{0≤t≤T:yt= h(t,xt)),h(t,x)=h(t,x)1{t<T}+g(x)1{t=T),and Introduce adjoint processes{Pt),{Qt}satisfying and If H(t,·,·,Pt,Qt,qt,·),h(t,·)and γ(·) are convex,and for any t∈[0,Ï„*],v∈U then u is an optimal control of Problem(P).Theorem 4.3.2. Let Assumption 4.1.1,4.1.2,4.1.3 and Assumption 4.3.2 hold.For any n∈N,suppose un is an admissible control and{xtn,0≤t≤T},{(ytn,ztn),0≤t≤T} are the corresponding solutions of (49) and(52),respectively.Denote Hn(t,x,y,P,Q,q,v)=<b(t,x,v),Q)+<σ(t,x),q>+<f(t,x,y,v)+n(y-h(t,x))-,P>. Introduce adjoint processes{Ptn},{Qtn}satisfying and If Hn(t,·,·,PtN,Qtn,qtn,·),γ(·) and g(·) are convex, and for any t∈[0,T], v∈U, then un is an εn-optimal control of Problem (P), where {εn}n≥1 decreases to 0 as nâ†'∞.4. Forward-Backward Stochastic Differential Equations in Weak Formulation.We introduce a new kind of forward-backward stochastic differential equations in weak formulation: From the viewpoint of both theoretical results and practical applications, we provide several specific examples, such as Example 5.1.2,5.2.1 and 5.2.2. for the motivations of studying weak FBSDEs, especially the connections with stochastic optimal control theory, and see that (65) associates to a quasi-linear parabolic PDE: Define solutions of (65) by: Definition 5.1.1. We call that(i) a filtered probability space (Ω,F,{Ft}0≤t≤T,P) and a quintuple of Ft-adapted pro-cesses{{Wt, Xt, Yt, Zt, Nt),0≤t≤T} is a weak solution of the weak FBSDE (65) if they satisfy (65), P-a.s., W is a P-standard Brownian motion, N is a P-martingale orthogonal to X with N0=0;(ii) a weak solution is semi-strong if (Y, Z) are FtX-adapted;(iii) a weak solution is strong if N= 0 and (X,Y, Z) are FtW-adapted.Moreover, by virtue of the related PDE (66), we obtain the wellposedness of weak FBSDE (65) as follows:Theorem 5.3.1. Let Assumption 5.3.1 hold. If the PDE (66) has a classical solution u∈C1,2 with uniformly bounded derivatives (?)xu and d(?)xx2u, then the FBSDE (65) admits a strong solution. In addition, if Assumption 5.3.2 and 5.3.3 hold, then the strong solution is unique.Theorem 5.3.2. Let Assumption 5.3.1,5.3.2 and 5.3.3 hold. If the PDE (66) has a viscosity solution u∈C0,0 and b, a do not depend on z, then the FBSDE (65) admits a semi-strong solution. Moreover, if u∈C0,1, then the FBSDE (65) also admits a semi-strong solution when b, σ depend on z explicitly.