Derivative Over Wasserstein Spaces Along Curves Of Densities And Its Applications To Mean-field Stochastic Control Problems

Stochastic control is a very important part of modern control theory.Our goal in a stochastic control problem is to observe the information and select an appropriate con-trol,so that a given functional of the state process in the dynamic can reach an optimal situation.For example,in the stock market,it can be regarded as a stochastic control problem to maximize the wealth value at a certain time by updating the portfolio at any time.There are two well-known classical methods to solve stochastic control problems,namely stochastic maximum principle(SMP)and dynamic programming principle(DP-P).The former obtains the necessary conditions for the optimal control,while the latter mainly adopts the idea from local to global and finds the optimal control by establishing a connection to partial differential equation.In this paper we mainly use the method of maximum principle.However,in most of the works on stochastic control,we assume that all the informa-tion can be known,which is to say,all the information of Brownian motion in the control system can be observed.It's easy to see that this assumption may not be reasonable.There are many times when we don't know all the information,but only a part of it.Therefore,partial-observed stochastic control systems have gradually entered the vision of researchers.The mean-field method in mathematics is more and more widely used in many fields,such as Economics,Finance,Physics and Quantum Chemistry.In recent years,many scholars have devoted themselves to the study of mean-field problems.The so-called mean-field stochastic differential equations,also known as Mckean-Vlasov equations and mean-field backward stochastic differential equations mean that the coefficients of the equations not only depend on the paths of the solutions of the equations,but also depend on the distributions of the solutions.The rapid development of mean field SDEs and BSDEs theory provides us with a powerful theoretical tool to study the problems of mean-field stochastic control.In this paper,we mainly study a type of mean-field stochastic control problems with partial observations.For the need of studying control problems later,we first consider the definition of differentiability of functionals defined on the set of Girsanov densities,under a given probability space.Thanks to the methods of Frechet derivatives,we provide a definition of the derivative with respect to the density,and show its connection with derivative of functions defined on P2(Rd).After getting the differentiability result,we can study a type of mean-field stochastic control problems with partial observations,driven by a group of "close-looped" conditional mean-field stochastic differential equations.We get the well-posedness of the dynamic and find a weak solution unique in law.Then by the duality,we obtain the stochastic maximum principle of optimal control,when the control domain is not necessarily convex and coefficients of the control system have no regularity on control term.Let us introduce the content and structure of this thesis.In Chapter 1,we introduce the main problems and their research backgrounds and motivations.In Chapter 2,we study such a function FQ:LQ?R defined by FQ(L):=f((LQ)?),L?LQ,where f:P(Rd)?R is given and ? is a given random vari-able.LQ denotes the collection of Girsanov densities on probability Q.We give the definition of differentiability of FQ.Then we prove that we can find a Borel measurable fuuction g:Rd?R to characterize its derivative,and the function g depends on(L,?)only through(LQ)?.Then we study the relations between the derivative with respect to the density and partial derivatives studied by[16].The novelty of this chapter is to give the definition of derivative with respect to the density for the first time and study its properties.Then we obtain its connections with partial derivatives.In Chapter 3,we furthermore study the derivative with respect to the density and build a connection between it and derivative of functions defined on P2(Rd).This chapter extend the result in Chapter 2.We prove that if f:P2(Rd)?R satisfies appropriate assumptions,there are connections between the derivative with respect to the density and the derivative of f with respect to probability measure,i.e.,the latter can be regarded as the derivative of the former.We first consider the 1-dimensional case,and prove the result for a special case that ? is a smooth Wiener functional.Then applying the approximation result,we generalise it to all the densities case.At the end of this chapter,we relate the mean-field approach based on densities of random variables(see,e.g.,[10])to our results.The novelty of this chapter is to generalise the result of derivative with respect to the density,and establish its connection with derivative with respect to probability measure.In the proof we have used the methods of Malliavin calculus and Girsanov transformation,which provide a new idea to solve similar problems.The Chapters 2 and 3 of the present paper are based on:R.Buckdahn,J.Li,H.Liang.Derivative over Wasserstein spaces along curves of densities.Submitted.In Chapter 4,we introduce a type of of mean-field stochastic control problems with partial observations.The stochastic dynamic is a pair of mean-field stochastic differential equations,one of which is state equation and the other is observation equation.The coeficients depends on the paths of observation process and the law of filtered state process knowing observation process in a nonlinear way.We first prove the existence and uniqueness in law of the weak solution to such a type of "close-looped" conditional mean-field SDEs,and then consider about the control problem.With the help of result in Chapter 3.we prove the first and second orders variation equations,and then duality principle helps us to give a necessary condition of optimal control.We get the Peng's stochastic maximum principle where the control domain is not necessarily convex and there is no regularity of coeficients on control.Through deriving the maximum principle,the variation equations and adjoint equations are new types of mean-field SDEs and BSDEs.The novelty of this chapter is to generalise the result in Buckdahn,Li and Ma[20]to a non-convex control space case.We prove a necessary condition of optimal control and get some new types of mean-field SDEs and BSDEs.The Chapter 4 of the present paper is based on:J.Li,H.Liang.A general mean-field stochastic maximum principle with partial observations.Preprint.This paper include four chapters,we now give an outline of the structure and the main conclusion of this dissertation.Chapter 1 Introduction;Chapter 2 Derivative with respect to the density;Chapter 3 Derivative with respect to the density and its connection with the derivative over P2(R);Chapter 4 Mean-field stochastic control problem with partial observations.Chapter 2:We study functions of densities FQ:LQ?R defined by FQ(L):=f((LQ)?),L?LQ,where function f:P(Rd)?R and random variable ? are arbitrarily given,LQ denotes the collection of Girsanov densities under given probability measure Q.We make the definition of differentiability of FQ,and prove that we can characterize the derivative by a Borel function g:Rd?R.Moreover,the function g depends on(Q,L,?)only through(LQ)?.Then we study the relations between the derivative with respect to the density and partial derivatives studied by[16].We define the space LQ:={L ? L1(?,F,Q)|L>0,EQ[L]=1}.Let??L0(?,F,Q;Rd)be arbitrarily given.For L?LQ,(LQ)?P(Rd)is defined by?Rd?d(LQ)?=EQ[?(?)L],??bB(Rd),where bB(Rd):={?:Rd?R|? bounded Borel function}.Let us fix now an arbitrary function f:P(Rd)?R,and put FQ(L):=f(LQ)?),L?LQ.(0.0.12)We give the definition of differentiability of function introduced by(0.0.12)as fol-lows,in the sense of the Frechet derivative over Banach spaces.Definition 2.2.1.Given L ?LQ,we say that FQ:LQ?R defined in(0.0.12)is differentiable at L,if there is some(DFQ)(L)? L(L01(Q,F,Q);R)such that FQ(L')-FQ(L)=(DFQ)(L)(L'-L)+o(|L'-L|L1(Q)),(0.0.13)for all L'?LQ with |L'-L|L1(Q)?0.We can prove that this definition is well-stated Lemma 2.2.2.For any given L?LQ,we suppose that the function FQ:LQ?R defined in(0.0.12)is differentiable at L in the sense of the above Definition 2.2.1.Then the continuous linear functional(DFQ)(L)?L(L01(?,F,Q);R)satisfying(0.0.13)is unique.Let us study now some properties of this derivative.For L ? LQ we put QL:=LQ.Note that QL is a probability on(?,F)and LQL={L'?L1(?,F,QL;R+):EQL[L'](=EQ[L'L])=1}.Lemma 2.2.3.Let L ? LQ.Then the function FQ:LQ?R is differentiable at L if and only if FQL:LQL?R is differentiable at L0=1.Moreover,if FQ:LQ?R is differentiable at L(and,thus,equivalently,FQL:LQL?R is differentiable at L0=1),then we haveDFQL(1)=DFQ(L)-EQL[DFQ(L)]?L0?(?,F,QL),QL-a.s.(?Q-a.s.),DFQ(L)=DFQL(1)-EQ[DFQL(1)]?L0?(?,F,Q),Q-a.s.Combining the above Lemma 2.2.3 with the following theorem,we can characterize the derivative by a bounded Borel function,and the function depends on(Q,L,?)only through(QL)?.Theorem 2.2.1.Let FQL:LQL?R be differentiable at L0=1.Then there exists a bounded Borel function g:Rd?R such that DFQL(1)=g(?),Q-a.s.Moreover,g depends on(Q,L,?)only through the law(QL)?.The above theorem justifies the following notation:(?)iF((QL)?,x):=9(x),x ? Rd.We observe that the function is(QL)?(dx)-a.s.well defined,and(?)1F(QL)?,?)=DFQL(1),QL-a.s.Then we consider of the relation between the derivative with respect to a density introduced in the preceding subsection and partial derivatives of functions over a suitable space of probability measures studied by[16].For L ?LQ?L2(?,F,Q),we define the function GQ,?(L)=G(Q(L,?))=f((LQ)?)=FQ(L),then we have the definition of partial differentiability as follows.Definition 2.3.1.The mapping G:P2,0(R×Rd)?R is said to be(partial)dif-ferentiable w.r.t.QL|? at Q(L,?),if GQ,?:L2(?,F,Q)?R is Frechet differentiable at L.Lemma 2.3.1.Let f:M(Rd)?R be given such that the function G:P2.0(R×Rd)?R defined for all probability Q by G(Q(L,?)):=f((LQ)?),(L,?)?L2(?,F,Q)×L0(Q,F,Q;Rd),is partial differentiable w.r.t.(QL)L'|? at(QL)(1,?)and FQL:LQL?R introduced in(2.2.1)is differentiable at L0=1.Then,(?)1F(QL)?,x)=((?)?G)1((QL)?,x)-EQL[((?)?G)1((QL)?,?)],x?Rd,(QL)?(dx)-a.s.Chapter 3:We further study derivative w.r.t.the density,and investigate the relation between the derivative w.r.t.the density and the derivative over P2(Rd)w.r.t.the probability measure.We prove that if f:P2(Rd)?R is differentiable,the derivative w.r.t.the density we discuss can be connected with derivative of f w.r.t.the probability measure and the latter can be regarded as a second order derivative.We first prove the result in a special case of smooth Wiener functional,then by the appropriate result we extend it to the general case.Later,we generalize the result from 1-dimensional case to multi-dimensional case,and relate the mean-field approach based on densities of random variables to the results in our paper.Let f:P2(Rd)?R be a continuously differentiable function,A(?)Rm a connected subset,and ?(?)??L??LQ ?L2(?,F,Q)continuously L2((Q)-differentiable.We put Q?=L?Q.We are going to consider about the differentiability of ?(?)??f(Q??)over A,and the derivative(?)?f(Q??).Theorem 3.1.1.Under Assumption 1,with the notation Q?:=L?Q,A ???,we have that the function ?(?)??f(Q??)is differentiable,and Theorem 3.1.2.Under Assumption 1,the function FQ(L):=f((LQ)?),L?LQ?L2(?,F,Q)is continuously L2(Q)-differentiable,DFQ(L)=?0?(?)?f((LQ)?,y)dy-EQ[?0?(?)?f((LQ)?,y)dy],Q-a.s.,L ?LQ?L2(?,F,Q),and,for the derivative at L'=1 of the function L'?FQL(L')=f((L'QL)?)(=f((L'LQ)?)),L'?LQL?L2(?,F,QL),DFQL(1)=?0?(?)?f((QL)?,y)dy-EQL[?(?)?f((QL)?,y)du],QL-a.s.,i.e.,(?)1F((QL)?,x)=?0x(?)?f((QL)?,y)dy-EQL[?0?(?)?f((QL)?,y)dy],(QL)?(dx)-a.s.,x?R.Moreover,(?)1F((QL)?,·):R?R is continuously differentiable,and(?)x((?)1F((QL)?,x)=(?)?f(QL)?,x),x?R.We prove Theorem 3.1.1 first for a special case.For simplicity we fix T=1.We suppose:Assumption 3.Let n? 1,0=t0<…<tn?1 ?i=(ti-1,ti],B(?i):=Bti-Bti-1.i)? is a smooth Wiener functional of the form:?=?(B(?1),...,B(?n)),?,?Cb?(Rn);ii)?? is a smooth Wiener step process of the form:where ?i:A ×Ri-1?R is a bounded Borel function,such that:iia)?i?:Ri-1?R is of class C? and all derivatives of all order are bounded over? ×Ri-1,1?i?n,andiib)A(?)?????LF2([0,1]×?,dsdQ)is continuously L2(dsdQ)-differentiable.For t ?[0,1]and ??A we introduce the Dolean-Dade exponential and we note that ?t??LQ?L?,-(Q),where L?,-(Q):=?1<p<+?Lp(Q).Moreover,we use the following notations:Qt?:=?t?Q and(Qt?)?=(?t?Q)??P2(R).Proposition 3.1.1.Under the Assumptions 2 and 3 the result stated in Theorem 3.1.1 holds true,i.e.,A(?)??f((Qt?)?)is differentiable and(?)?f((Qt?)=EQ[(?0?(?)?f((Qt?)?,y)dy)(?)?[?t?]].The proof of Proposition 3.1.1 is heavily based on Girsanov transformation and Malliavin calculus.The following Proposition 3.1.2 allows us to translate the derivative w.r.t.the density of ?'? f((?t?'Q)?)at ?'=? to a derivative w.r.t.the law of?'?f((Qt?)?(Tt?,(B?)))).Proposition 3.1.2.Given any function f:P2(R)?R,we have f((?t?'Q)?)-f((?t?Q)?)=f((?t?,?'Qt?)?)-f((Qt?)?)=f((Qt?)?(Tt?,?'(B?)))-f((Qt?)?(B?)),?'?A.We also need the following result of Malliavin calculus to prove the special case.Lemma 3.1.2.Under Assumption 3 we have that the mapping s??(Ts?',?(B?))is continuous on[0,1]and differentiable in all s?(ti-1,ti),1 ?i ?n:(?)s[?(Ts?',?(B?))]=(Ds?)(Ts?,?'(B?))·(?s?'-?s?)(Ts?,?'(B?)).The following approximation result is used to prove the the main statement in this chapter-Theorem 3.1.1.Proposition 3.1.3.Let ?(?)?? L? ?LQ ? L2(?,F,Q)be a continuously L2(Q)-differentiable mapping.Then there exists a sequence of bounded smooth Wiener step processes ??n,n?1,#12 with 0=t0n<t1n<...<fNnn=1,?in=(ti-1n,tin],such that?)?in:A×Ri-1?R is a bounded Borel function,?)?i?,n:Ri-1?R is of class C? and the derivatives of all order are bounded over? × Ri-1,1?i?n,?)?(?)????,n?LF2([0,1]× ?,dsdQ)is continuously L2(dsdQ)-differentiable on?,such that,for ?t?,n:=exp {?0t ?s?,ndBs-1/2?0t|?s?,n|2ds},ds},t[[0,1],??A,we have a)(?1?,n,(?)??1s,n)(?)(L?,(?)?L?),for all ??,b)for all ?,?'?A with[?,?'](?)?,?(s):=s?'+(1-s)?,s?[0,1],(?1?(·),n,(?)??1?(·).n)(?)(L?(·),(?)?L?(·)).In the discussion of multi-dimensional case,we obtain the following result.Theorem 3.2.1.Let L ? LQ be such that,for some real constants C,c>0,c<L<C,and suppose that,for all ??L4(?,F,Q;Rd)and for f:P2(Rd)?R satisfying As-sumption 1 we have that the function FQ?(L'):=f((L'Q)?),L'?LQ?L2(?,F,Q),is L2(Q)-differentiable,DFQ?(·)is L2(Q)-Lipschitz,(?)1F((QL)?,·)is continuously differen-tiable,and(?)x((?)1F)(·,·),(?)?f(·,·):P2(Rd)×Rd?R are bounded and continuous(with continuity modulus).Then,(?)x((?)1F)((QL)?,x)=(?)?f((QL)?,x),x ?Rd.At the end of this chapter,we consider functions of the form f((LQ)?)=?(f?LQ)defined as a differentiable function ? of the density of the law of a random variable??L2(?,F,LQ).We get a similar result as above theorem in this setting.Chapter 4:We introduce a type of mean-field stochastic control problems with partial observations.First we get the well-posedness,i.e.,existence of the weak solution and its uniqueness in law,of the dynamic.When consid-ering control problems,we apply the results about derivative from Chapter 3 to write and to prove the variation equations.Then we obtain a necessary condition for optimal control thanks to the duality principle.Our result here is Peng's stochastic maximum principle,when it does not need the convexity of control space and the regularity of coefficients w.r.t.the control.In this chapter,we consider of the following state-observation dynamics:where(B1,B2)is an(F,P)-Brownian motion.In the above system,X is the state pro-cess,while Y is the observation process,defined on(?,F,P).Let UtX|Y:=EP[Xt|FtY],t?[0,T],denote the "filtered" state process and ?tX|Y its law under P,i.e.,?tX|Y:=PUtX/Y.We first consider its well-posedness under suitable assumptions.From Girsanov Theorem,we can transfer(0.0.14)to the following form:This equation can be regarded under referelce probability measure Q,where(B1,Y)is an(F,Q)-Brownian motion.Proposition 4.1.1.Under(H1)equation(0.0.15)possesses a unique strong solution.The existence of a strong solution(X,L)of SDE(0.0.15)implies that of a weak solution of(0.0.14),i.e..(?,F,F,P,(B1,B2),(X,Y))is a weak solution of(0.0.14),where P=LTQ,and Bt2=Yt-?0th(s,Y.?s,Xs,?sS|Y)ds,t?(0,T].We also have the following result of uniqueness.Theorem 4.1.1.Under Assumption(H1),let(?i,Fi,Fi,Pi,(B1,i,B2,i),(Xi,Yi)),i=1,2,be two weak solutions of(0.0.14).Then it holds that P((B1,1,B2,1),(X1,Y1)1=P((B1,2,B2,2),(X2,Y2))2.Then we consider the control system under reference probability measure Q:where Pu=LTuQ,?tu=?tXu|Y=PEuu[Xtu|FtY]J and Eu[·]:=EPu[·].The cost functional is defined by J(u):=EQ(?(XT,u?Tu)+?0Tf(t,Xtu,?tu,ut)dt],u?uad.The goal of the control problem is to minimize the cost functional.In our discussion the control domain is not necessarily convex.Under appropriate assumptions,applying the result given in Chapter 3,we can get the following first order variation equation:We have the well-posedness of(0.0.17).Proposition 4.2.1·Under Assumption(H2),(0.0.17)has a unique solution(Y1,?,K1,?)?SF2([0,T]Q)×SF2([0:T],Q).Moreover,Y1,?,K1,?,V1,??SFp([0,T],Q)for all p?1.The following estimates also justify the correctness of variation equation.Proposition 4.2.2.For all k?1,there exists Ck ?R+,such that,(?)(?)(?)(?)(?)(?)(?)(?)Corollary 4.2.1.For all k?1,there exists Ck?R+,such that,(?)(?)(?)(?)(?)(?)(?)(?)The next result is very subtle and useful.Proposition 4.2.3.For all ?=(?1,?2)?LF2([0,T],Q;R2)with EQ(?(?t1|2+|Lt?t2|2)dt]<+?,and(?t1,Lt?t2)?L2(Ft,Q;R2)for all t ?[0,T],there exists ?:[0,T]× R+?R+ such that|EQ[?t1Yt1,?+?t2Kt1,?]??t(?)(?),??(0,1],t?[0,T],with pt(?)?0(?(?)0),t ?[0,T],and?t(?)?CEQ[|?t1|2+|Lt?t2|2],??(0,1],t?[0,T].The second order variation equation has the following form.For simplicity,we assume ?=?(?,v),h=h(x,v).where We have the following well-posedness and estimates for second order variation pro-cesses.Lemma 4.2.1.Under Assumption(H2),the equation(0.0.18)has a unique solution(Y2,?,K2,?)?SF2([0,T)×SF2([0,T],Q).Moreover,Y2,?,K2,? ? SF?-([0,T],Q)with SFp([0,T],Q)-bounds independent of ?>0,for all p>2.Lemma 4.2.2.For all p>1,there is a constant Cp ?R+ such that for t ?[0,T],?>0,(EQ[|(Ut?-(Ut-Vt1,?+Vt2,?))-?t(Xt?(Xt-Yt1,?+Yt2,?),Lt?(Lt-Kt1,?+Kt2,?))p]1/p?Cp?3/2.Proposition 4.2.4.For all p>2,there exists Cp?R+,such that,(?)(?)(?)(?)(?)(?)with pp(?)?0,as ?(?)0.Moreover,(?)(?)We emphasize that above results hold true also for general case,the dependentness of other variables will not make the computations more difficult.We consider the duality now,and get the first order adjoint equation.whereProposition 4.2.6.Under the Assumption(H2),BSDE(0.0.19)has a unique strong solution((p1,(q1,q1)),(p2,(q2,q2))).Moreover,for any p>2,it holds that((p1,(q1,q1)),(p2,(q2,q2)))?(SFp([0,T],q)(LFp([0,T],Q))2)×(SF2p([0,T),Q)×(LF2p([0,T],Q))2).Let us define the Hamilton functional H(t,x,l,?,v,q1,q2):=?(t,x,?,v)q1+h(t,x,?,v)lq2-(t,x,?,v),(0.0.20)(t,x,l,?,v,q1,q2)?[0,T]×R×+×P2(R)×R×R,and notations?H(t)=??(t)qt1+?h(t)Ltqt2-?f(t),Hxx(t)=?xx(t)qt1+hxx(t)Ltqt2-fxx(t),Hxl(t)=hx(t)qt2,H?(t)=??(t)qt1+h?(t)Ltqt2-f?(t),Hz?(t)=?z?(t)qt1+hz?(t)Ltqt2-fz?(t),where((p1,(q1,q1)),(p2,(q2,q2)))is the solution of first order adjoint equation.Thenwe give the second order adjoint equation.Now we can conclude our stochastic maximum principle.Theorem 4.2.1.Assume the Assumption(H2).Let u?uad be optimal and(X,L)be the associated solution of system(0.0.16).Then,for all v?U,it holds that for dtdQ-a.e.(t,?)?[0,T]×?,where((p1,(q1,q1)),(p2,(q2,q2)))and((P1,(Q1,1,Q1,2)),(P2,(Q2,1,Q2,2)))are the u-nique solutions to(0.0.19)and(0.0.21),respectively.