| In this thesis,delayed stochastic optimal control problems and stochastic Stackelberg differential games are studied.We establish the global maximum principle,the relationship between maximum principle and dynamic programming principle,the verification theorem,also derive the feedback control,the open loop strategy,and discuss the closed-loop solvability.We also apply these theoretical results to practical problems,such as optimal investment problems,production and consumption problems and resource allocation problems.In the fields of finance,aerospace,network communication and so on,many problems can be modeled as the optimal control problems.However,the development of some phenomena in the real world depends not only on the current state,but also on the historical state in the past.For example,due to the limited speed of signal transmission,the time required to travel a long distance communication is not negligible,or the time required to travel through a queue is not negligible.In hydraulic control systems,non-instantaneous human or chemical reactions can cause time delays,as can the viscoelastic effects of materials.Therefore it has very important theoretical significance and application value to study the delayed optimal control problems.In real life,we should not only optimize our own index functional according to our own information and our own behavior,but also interact with others’ behavior under the influence of others’ information to make more favorable decisions.A Stackelberg differential game is a game that describes players with unequal information or status.In a Stackelberg differential game,there are two players,one is the leader and the other is the follower.When the leader knows the rational response of the follower and reveals his strategy first,the follower does not know the rational response of the leader and must optimize his strategy under any given control of the leader.The Stackelberg strategy is the rational solution of the two players.Therefore,it has profound theoretical value and great practical significance to study the delayed Stackelberg differential games.Next,we give the main research content and specific organization structure of this thesis.In Chapter 1,we introduce the research background,the research purpose,the research significance,the research status,the research content,the innovation and contributions of the following five chapters.In Chapter 2,we study a delayed stochastic optimal control problem,where the control domain is nonconvex and the diffusion term contains both control and its delayed term.Inspired by previous results about delayed stochastic control systems,Peng’s global stochastic maximum principle is generalized to the time delayed case,the maximum condition consists of two parts:one is without delay while the other is with delay.The former is similar to Peng’s global maximum principle,while the latter is characterized by the conditional expectation、the indicator functions and the anticipated backward stochastic differential equations.A special backward stochastic differential equation is introduced to deal with the cross terms,when applying the duality technique.Furthermore,to illustrate the applications of our theoretical results,three dynamic optimization problems are addressed based on the global maximum principle.In Chapter 3,we focus on the stochastic recursive optimal control problem with mixed delay.The connection between Pontryagin’s maximum principle and Bellman’s dynamic programming principle is discussed.Without containing any derivatives of the value function,relations among the adjoint processes and the value function are investigated by employing the notions of super-and sub-jets introduced in defining the viscosity solutions.Stochastic verification theorem is also given to verify whether a given admissible control is really optimal.In Chapter 4,we investigate a linear quadratic optimal control problem of delayed backward stochastic differential equation.An explicit representation is derived for the optimal control,which is a linear feedback of the entire past history and the expected value of the future state trajectory in a short period of time.To obtain the optimal feedback,a new class of delayed Riccati equations and delayed-advanced forward-backward stochastic differential equations are introduced.Furthermore,the unique solvability of their solutions are discussed in detail.In Chapter 5,we discuss a linear quadratic stochastic Stackelberg differential game with time delay.The model is general,in which the state delay and the control delay both appear in the state equation,moreover,they both enter into the diffusion term.By introducing two Pseudo-Riccati equations and a special matrix equation,the state feedback representation of the open-loop Stackelberg strategy is derived,under some assumptions.Finally,two examples are given to illustrate the applications of the theoretical results.In Chapter 6,we consider a general time-varying linear quadratic optimal control problem with state delay,and not only the pointwise delay but also the distributed delay are involved.By lifting the delayed control problem in finite dimensional space to an infinite dimensional control problem without delay,we give two appropriate definitions:the open-loop solvability and the closed-loop solvability for the delayed control problem.The closed-loop solvability is characterized by three equivalent integral operator-valued Riccati equations and two equivalent backward integral evolution equations.The above results are not explicit because they are stated by many operators.Furthermore,we give the corresponding straightforward representations by coupled matrix-valued Riccati equations and coupled partial differential equations.Next,we give the main conclusions of this thesis.1.A global maximum principle for stochastic optimal control problems with delaySuppose U ?Rk is nonempty and nonconvex,let δ>0 be a given constant time delay parameter,we consider the following stochastic control system with delay:along with the cost functional(?)(0.0.55)We define the admissible control set as follows:uad:=｛v(·)|v(·)is a U-valued,square-integrable,Ft-predictable process｝.Problem(P).Our object is to find a control u(·)over uad such that(0.0.54)is satisfied and(0.0.55)is minimized.The following lemma gives the estimates of(2.3.2)and(2.3.3).Lemma 0.1.Let assumption(A2.1)hold.Suppose x(·)is the optimal trajectory,xε(·)is the trajectory corresponding to uε(·),then for any p≥1,(?)(0.0.56)(?)(0.0.57)(?)(0.0.58)(?)(0.0.59)(?)(0.0.60)The following lemma gives the variational inequality.Lemma 0.2.Let assumption(A2.1)hold.Suppose(u(·),x(·))is an optimal pair,xε(·)is the trajectory corresponding to uε(·)by(2.3.1),then the following variational inequality holds:(?)(0.0.61)By Lemma 0.1 and Lemma 0.2,we derive the global maximum principle.Theorem 0.1.Let assumption(A2.1)hold.Suppose(u(·),x(·))is the optimal pair.Let lxδ(t)=lxδxδ(t)=0 hold for t∈(T,T+δ].Suppose(p(·),q(·)))∈SF2([0,T];R)×LF2([0,T];R)and((P(·),Q(·)∈SF2([0,T];R)×LF2([0,T];R)satisfy the ABSDEs(2.3.12)and(2.3.13),respectively.Besides,suppose(K(·),K(·))satisfies the BSDE(2.3.14)with K(t)=K(t)=0 for all t ∈[0,T].Then the following maximum condition holds:(?)(0.0.62)where H is the Hamiltonian function defined by(2.3.25).2.Stochastic recursive optimal control problem with mixed delay under viscosity solution’s frameworkLet δ>0 be fixed,for any(s,φ)∈(0,T)× C([-δ,0];R),we consider the following controlled coupled forward-backward stochastic differential equation with mixed delay(FBSMDDE):where(?)(0.0.64)The cost functional is introduced as follows:J(s,φ;u(·)):=-Ys,φ;u(s),(s,φ)∈[0,T)× C([-δ,0];R).(0.0.65)Problem(P).For given(s,φ)∈[0,T]x C([-δ;R),our object is to find u*(·)∈uω[s,T],such that(0.0.63)admits a unique solution and(?)(0.0.66)The following theorem gives the connection between adjoint variables and the value function:Theorem 0.2.Let(H3.1),(H3.2)hold.Let(s,x,x1)∈[0,T]× R2 be fixed.Suppose that u*(·)is an optimal control,and the triple(X*(·),Y*(·),Z*(·))is the corresponding optimal trajectory.V(·,·,·)∈ C([0,T]×R2;R)is the value function.Letγ(·)∈ SF2([s,T];R),(p(·),q(·))∈SF2[s,T];R3)× LF2([s,T];R2)satisfy the adjoint equations(3.2.8),(3.2.9),respectively.Furthermore,assume that p3(t)≡0 for all t ∈[s,T].Then(?)(0.0.67)here X1*(t):=∫-δ0eλτX*(t+τ)dτ,X2(t):=X*(t-δ).We consider the special version of FBSMDDE(0.0.63):The following theorem gives the verification theorem of Problem(P)associated with(0.0.68).Theorem 0.3.Suppose b,σ,φ satisfy(H3.1),a,Φ satisfy(H3.2),g(·)≡ 0 and f(·)is a given uniformly bounded deterministic function.Let V(·,·,·)∈C([0,T]× R2;R),depending on(s,x,x1)only,be the viscosity solution to the following HJB equation satisfying |V(t,x,x1)|≤C(1+|x|k+|x1|k),for some k≥1,(t,x,x1)∈(0,T)× R2:where the generalized Hamiltonian function G:[0,T]×R×R×R×U×R×R×R×R→R is defined as(?)(0.0.69)then we have V(s,x,x1)≤ J(s,φ;u(·)),for all(s,φ)∈[0,T]× C([-δ,0];R),u(·)∈ uw(s,T).(0.0.70)Furthermore,let(s,x,x1,x2)E[0,T]× R3 be fixed,suppose(X*(·),Y*(·)Z*(·),u*(·))is an admissible pair such that there exists a quadruple(Θ,p,q,P)∈LF2([s,T];R)×LF2([s,T);R)×LF2([s,T];R)× LF2([s,T];R)satisfying(Θ,p,q,P)∈Dt+,x1,2,1,+V(t,X*(t),X1*(t),a.e.t ∈[s,T],P-a.s.,(0.0.71)and(?)(0.0.72)Then(X*(·),Y*(·),Z*(·),u*(·))is the optimal pair.Theorem 0.4.Suppose in Theorem 0,3,g(·)is any given uniformly bounded deterministic function,the corresponding HJB equation becomes(3.4.2)and(3.4.7)is given in the following form:In this case we can still obtain the same conclusion as Theorem 0.3.3.Linear quadratic optimal control problems of delayed backward stochastic differential equationsFor given s ∈[0,T),let us consider the following controlled linear delayed backward stochastic differential equation:along with the cost functional(?)(0.0.74)The admissible control set is defined as follows:u[s,T]:={u:[s-δ,T]× Ω→Rd| for t ∈[s-δ,s),u(t)=η(t);for t ∈[s,T],u(t)is an Ft-predictable process,E ∫sT|u(t)|2dt<∞}.Problem(D-BSLQ).For any s∈[0,T),ξ∈L2(Ω,FT,P;Rn),to find a u*(·)∈U[s,T]such that.(?)(0.0.75)Now we give a result to guarantee the well-posedness of(0.0.73).Theorem 0.5.Let(A4.1),(A4.2)hold and δ be sufficiently small.Then for any(ξ,u(·))∈ L2(Ω,FT,P;Rn)×u[s,T],the state equation(0.0.73)admits a unique adapted solution(Yu(·),Zu(·))∈SF2([s,T];Rn)×LF2([s,T];n).Moreover,there exists a constant K>0,independent of ξ and u(·),such that(?)(0.0.76)The following theorem gives the optimal control of Problem(D-BSLQ).Theorem 0.6.Let(A4.1)-(A4.3)hold and δ be sufficiently small.Assume Q(t)=R(t)=N(t)=0 fort ∈[T,T+δ],suppose u*(·)is an optimal control and(Y*(·),Z*(·))is the corresponding optimal state trajectory.Then the following equation admits a unique solution X*(·)∈SF2([s,T];Rn):and the optimal control can be expressed as u*(t)=N-1(t)C(t)TX*(t)+EFt[(CTX*)[tδ)]},a.e.t ∈[s,T],P-a.s.(0.0.78)The following theorem gives the feedback of Problem(D-BSLQ).Theorem 0.7.Let(A4.1)-(A4.3)hold and 8 be sufficiently small.Suppose ∑(·)∈C([s,T];S+n),L(·)∈C([s,T];S+n),S(·)∈SF2([s,T];Rn),(X(·),Λ(·),Γ(·))∈SF2([s,T];Rn)×SF2([s,T];Rn)×LF2([s,T];are the solutions to(4.3.1),(4.3.3),(4.3.5),(4.3.4),respectively.Assume that A(t)=B(t)=C(t)=0 for t ∈[s,s+δ],Q(t)=R(t)=N(t)=A(t)=B(t)=C(t)=0 for t∈[T,T+δ]and(?)(0.0.79)Then the optimal control can be expressed as(?)(0.0.80)Moreover,the optimal cost is(?)(0.0.81)To study the unique solvability of(4.3.1),(4.3.3)and(4.3.4)in the above theorem,first we consider the delayed Riccati equation(4.3.1),to this end we consider a general delayed Riccati equation:The following proposition gives the uniqueness of the solution to the delayed Riccati equation(0.0.82).Proposition 0.1.Let(A4.1),(A4.3)hold and δ be sufficiently small.If the solution E(·)to the delayed Riccati equation(0.0.82)satisfies ∑(·)∈C([s,T];S+n),then the solution is unique.If we can prove the existence of the solution to the delayed Riccati equation(0.0.82),then we can obtain its unique solvability.However,this is an arduous work and we can only deal with the special case so far in this paper.Let R(·)+R(·+δ)≡ 0,now(0.0.82)becomes(4.4.5).The following proposition gives the unique solvability of(4.4.5).Proposition 0.2.Let(A4.1),(A4.3)hold and δ be sufficiently small.Let M≥0 be n× n symmetric matrix,suppose(A4.4)holds and B(·)=I,then(4.4.5)has the unique solution ∑(·)∈ C([s,T]S+n).Furthermore,suppose(A4.5)holds and M>0,then ∑(·)∈ C([s,T];S+n).From the above proposition,in Theorem 0.7 the delayed Riccati equation(4.3.1)admits the unique solution.Corollary 0.1.Let(A4.1),(A4.3),(A4.4)hold and δ be sufficiently small.Let B(·)=I,R(·)+R(·+δ)=0,then(4.3.1)admits the unique solution ∑(·)∈C([s,T];S+n).The following proposition gives the unique solvability of the delayed Riccati equation(4.3.3)in Theorem 0.7.Proposition 0.3.Let ∑(·)be the solution to(4.3.1),then the Riccati equation(4.3.3)is uniquely solvable,and(i)if G>0,then L(·)∈ C([s,T];S+n);(ii)if G≥ 0,then L(·)∈ C([s,T];S+n).Before considering the existence and uniqueness of the solution to the equation(4.3.4),we would like to study the following general equation:Then the following theorem gives its unique solvability.Theorem 0.8.Let(H4.5)-(H4.8)hold and δ be sufficiently small,and suppose either L4,L6 are sufficiently small,or L2,L8 are sufficiently small,then for any ζ∈L2(Ω,FT,P;Rn),the general equation(0.0.83)has a unique solution(X(·),Λ(·),Γ(·))∈SF2([s,T];Rn)× SF2([s,T];Rn)×LF2([s,T];Rn).By Theorem 0.8,we get the following result immediately.Corollary 0.2.Let(A4.1),(A4.3)hold and δ be sufficiently small.And assume one of the following two conditions holds:(i)(?)is sufficiently small;(ii)(?)is sufficiently small.s<t<T s<t<T Then the equation(4.3.4)has a unique solution(X(·),Λ(·),Γ(·))∈SF2([s,T];Rn)×SF2([s,T];Rn)× LF2([s,T];Rn).4.A linear quadratic stochastic Stackelberg differential game with time delay Consider the following linear controlled system with time delay:and the cost functionals for the leader and the follower are as follows,respectively:(?)(0.0.85)In the above,X(·)∈ Rn is the state process,u1(·)∈Rk1 and u2(·)∈ Rk2 are the control processes of the follower and the leader,respectively φ(·)∈C([-δ,0];Rn)is the initial trajectory of the state,ηi(·)∈L2([-δ,0];Rki),i=1,2,are the initial trajectories of the follower’s and the leader’s control,respectively.Our linear quadratic stochastic Stackelberg differential game with time delay is the following.For each choice of the leader u2(·)∈ u2[0,T]:=LF2([0,T];Rk2),the follower would like to choose a strategy u1[0,T]:=LF2([0,T];Rk1)such that his cost functional J,(u1(·);u2(·))is the minimum of J1(u1(·),u2(·))over u1(·)∈u1[0,T].The leader has the ability to know the optimal strategy of the follower u1(·),thus the leader would like to choose a strategyr u2(·)∈u2[0,T]such that his cost functional J2(u1(·),u2(·))is the minimum of J2(u1(·),u2(·))over u2(·)∈u2[0,T].Strictly speaking,the follower wants to find a map α1[·]:u2[0,T]→u1[0,T]and the leader wants to find a control u2(·)∈u2[0,T],such thatWe deal with the optimization problem of the follower,which is a linear quadratic stochastic optimal control problem with time delay,for any choice u2(·)of the leader.Problem(F-DLQ):For any u2(·)∈u2[0,T],minimize the cost functional J1(u1(·),u2(·))over u1(·)∈u1[0,T]such that(0.0.84)is satisfied.For any given admissible control u2(·)∈u2[0,T],the following theorem gives thesufficient and necessary conditions of the solvability for Problem(F-DLQ).Theorem 0.9.Let(A5.1)hold,and Q1(t)=R1(t)=0 for t ∈(T,T+δ].Let P1(·)satisfy(5.2.4)and(ζ1(·),ζ1(·))satisfy(5.2.5).Then for any u2(·)∈u2[0,T],Problem(F-DLQ)is solvable and the optimal control u1(·)is of the following state feedback form:(?)(0.0.86)Moreover,the optimal cost can be expressed as(5.2.7).Next we will address the optimization problem of the leader.Problem(L-DLQ):Minimize the cost functional J2(u1(·),u2(·))over u2(·)∈u2[0,T]such that(5.3.1)is satisfied.The following theorem gives the necessary condition of the solvability for Problem(L-DLQ):Theorem 0.10.Let(A5.1)-(A5.3)hold,assume Qi(t)=Ri(t)=0 for t∈[T,T+δ],i=1,2.Suppose the matrix equation(5.3.10)and(5.3.11)have the unique solutions L(·)and ∏(·,·),respectively.Let u2(·)be the optimal control of the leader and X(·)be the optimal state strategy for Problem(L-DLQ),then u2(·)is of the following state feedback form:(?)(0.0.87)Moreover,the optimal cost of the leader can be expressed as(5.3.27).Finally we summarize the above contents and state the main result for the linear quadratic Stackelberg differential game with time delay.Theorem 0.11.Let(A5.1)-(A5.3)hold,assume Qi(t)=Ri(t)=0 for t ∈[T,T+δ],i=1,2.Suppose(u1(·),u2(·))is the optimal open-loop strategy and the matrix equation(5.3.10)and(5.3.11)have the unique solutions L(·)and ∏(·,·),respectively,then the optimal open-loop strategy is given by(?)(0.0.88)u2(t-δ)=Ku2(t-δ)Φ(t|t-δ),a.e.t∈[δ,T+δ],P-a.s.,(0.0.89)here Φ(t|t-δ),Ku2(t-δ),K1u1(t-δ),K2u1(t-δ)are defined as(5.3.30).5.Time-delayed linear quadratic optimal control problems:closed-loop solvabilityFor given s∈[0,T)and a constant time delay δ>0,we consider the following controlled linear ordinary differential delayed equation:In the above X(·)is the state and u(·)∈L2([s,T];Rm)is the control.∫[-δ,0]A0(t,dθ)X(t+θ)represents the pointwise delay and the distributed delay of state,where(?)(0.0.91)with-δ=θN<θN-1<…<θ1<0.φ0 is the initial state,φl(·)∈ L2([-δ,0]:Rn)is the initial trajectory of the state.Next consider the following cost functional:(?)(0.0.92)here the above terms are defined as(6.1.4)-(6.1.10).Problem(P).For any(s,φ0,φ1)∈[0,T)x M2,to find a u*(·)∈ L2([s,T];Rm)such that(0.0.90)is satisfied and(?)(0.0.93)First we transform Problem(P)into the LQ problem without delay,which is formulated specifically as follows.Problem(EP).For any(s,φ)∈[0,T)× M2,to find a u*(·)∈ L2([s,T];Rm)such that the following state equation is satisfied:(?)(0.0.94)and(?)(0.0.95)where M2:=Rn × L2([-δ,0];Rn),and(?)(0.0.96)Next we shall study the closed-loop solvability for Problem(P)by studying Problem(EP),thus we introduce the following definitions.Definition 0.1.(1)Problem(P)is said to be(uniquely)open-loop solvable at initial pair(s,φ0,φ1)∈[0,T]×M2 if there exists a(unique)u*(·)∈L2([s,T];Rm)satisfying(0.0.95).(2)Problem(P)is said to be(uniquely)open-loop solvable at some s ∈[0,T)if for any(φ0,φ1)∈M2,there exists a(unique)u*(-)∈ L2([s,T];Rm)satisfying(0.0.95).(3)Problem(P)is said to be(uniquely)open-loop solvable on[s,T)if it is(un.iquely)open-loop solvable at all t ∈[s,T).Definition 0.2.(1)LetΞ[s,T]:=｛Θ(·):[s,T]→L(M2,Rm)|Θ(·)is strongly continuous and(?)and Q[s,T]:=Ξ[s,T]× L2([s,T];Rm).Any pair(Θ(·),v(·))∈Q[s,T]is called a closedloop strategy of Problem(P)on[s,T].(2)For any(Θ(·),v(·))∈Q[s,T]and(φ0,φ1)∈M2,letφ=(φ1 φ0),X(·)≡X(·;s,φ,Θ(·),v(·))be the solution to the following equation:(?)(0.0.97)and let u(t)=Θ(t)X(t)+v(t),t ∈[s,T],then(X(·),u(·))is called the outcome pair of(Θ(·),v(·))on[s,T]corresponding to the initial trajectory(φ0,φ1);X(·),u(·)are called the corresponding closed-loop state and closed-loop outcome control,respectively.(3)A closed-loop strategy(Θ*(·),v*(·))∈ Q[s,T]is said to be optimal on[s,T]if(?)(0.0.98)where X*(·),X(·)are the closed-loop states corresponding to(Θ*(·),v*(·),φ0,φ1),(Θ(·),v(·),φ0,φ1),respectively.If an optimal closcd-loop strategy(uniquely)exists on[s,T],Problem(P)is said to be(uniquely)closed-loop solvable on[s,T].The following lemma gives the sufficient and necessary conditions of the open-loop solvability for Problem(P).Lemma 0.3.Let(A6.1)-(A6.2)hold.For any given initial pair(s,φ0,φ1)∈[0,T)×M2,u*(·)is an open-loop optimal control of Problem(P)if and only if the following two conditions hold:(i)(Stationarity condition)S0(t)X*(t)+R00(t)u*(t)+B(t)*p*(t)+ρ0(t)=0,a.e.t ∈[s,T],(0.0.99)where(X*(·),p*(·))E C([s,T];M2)× L∞([s,T];M2)satisfies the following integral equation:(0.0.100)(ii)(Convexity condition)(?)(0.0.101)where X0(·)is the solution to the following integral equation:(?)(0.0.102)We derive the sufficient and necessary conditions of the closed-loop solvability for Problem(P)by virtue of Problem(EP).Theorem 0.12.Let(A6.1)-(A6.3)hold.Suppose B,R,R? are continuous,S00(t,θ),S01(t,θ))are continuous in t,uniformly in θ∈[-δ,0],S0*[I-R00?R00]=0,then(Θ*(·),v*(·))is the optimal closed-loop strategy of Problem(P)on[s,T]if and only if(i)(Θ*(·),v*(·))is given by(6.3.44),here for any φ∈M2,P(·)satisfies the integral Riccati equations(6.3.41)and η(·)satisfies the integral equations(6.3.42),(ii)R00>0,R(B*P+S0)? R(R00),R(B*η+ρ0)?R(R00),(iii)(e*(·),v*(·))∈Q[s,T].In the case the value function is(?)(0.0.103)Noting that the above results are stated by many operators,thus we give a more straight expression of the closed-loop solvability for Problem(P).Theorem 0.13.Let(A6.1)-(A6.2)hold.Assume R≥0,B(t),S00(t),S01(t,θ),R(t),R(t)t are continuous in t.Let P00(t),P01(t,θ),P11(t,θ,α),η(t),η1(t,θ),t ∈[s,T],θ,α∈[-δ,0],be absolutely continuous functions satisfying the equations(6.3.87)-(6.3.88),and P00(t)=P00(t)T,P11(t,θ,α)=P11(t,α,θ)T,and(?)(0.0.104)Let(Θ*(·),v*(·))be given by(?)(0.0.105)Then(Θ*(·),v*(·))is the optimal closed-loop strategy and the value function is(?)... |
PDF Full Text Request