| As the mainstay of gravitational theory, Einstein’s general relativity provides precise descriptions to a variety of phenomena in our Universe. But, it is also well known that general relativity suffers various troublesome theoretical problems, such as the singularity problem, the nonrenormalization problem and the dark matter/energy problem. Thus, modified gravity theories are helpful to enrich our knowledge about the gravity and may provide us new approaches to overcome these problems.On the other hand, extra dimensions are usually introduced to unify the gravity and other interactions. In order to address the problem of why physical properties of the observed four-dimensional spacetime are totally different from the extra dimensions, the braneworld scenario as one of the possible mechanisms was suggested. It has been inten-sively investigated during recent years. However, since the majority of current braneworld models are based on the general relativity, which is widely believed to need modifications in ultraviolet regime, it is very interesting and quite necessary to generalize the braneworld scenario in modified gravity theories.There are four parts in this thesis. In the first part, we simply introduce the back-ground of our research-alternative gravity theories and braneworld theory.In the second part, we investigate two modified gravity theories.Firstly, we investigate the linear perturbational stability in Eddington-inspired Born-Infeld gravity, which put forward by M. Banados and P. G. Ferreira. Since the homoge-neous and isotropic cosmology seems to be singularity free in this theory, it has drawn much attention. We study the full linear perturbations of a homogeneous and isotropic spacetime in the Eddington-inspired Born-Infeld gravity. The stabilities of the perturba- tions are analyzed in the Eddington regime. We find that, for positive b, the scalar modes are stable in the infinite wavelength limit{k= 0) but unstable for k ≠0. The vector modes are stable and the tensor mode is unstable in the Eddington regime, independent of the wave vector k. However, these modes are unstable and hence cause the instabilities for negative b.Secondly, we generalize the theory of emergent cosmic space. A novel idea about our expanding Universe was proposed by Padmanabhan, recently. He suggested that the expansion of our Universe can be thought of as the emergence of space as cosmic time progresses. The emergence is governed by the basic relation that the increase rate of Hubble volume is linearly determined by the difference between the number of degrees of freedom on the horizon surface and the one in the bulk. So following this idea, we generalize the basic relation to derive the Friedmann equations of an (n+1)-dimensional Friedmann-Robertson-Walker universe corresponding to general relativity, Gauss-Bonnet gravity, and Lovelock gravity.In the third part, we consider braneworld models based on three alternative gravies and investigate their physical properties, respectively.Firstly, we consider a braneworld model in the scalar-tensor gravity. In order to solve the gauge hierarchy problem in this model, our world should be confined on the positive tension brane rather than on the negative one. This is crucial to reproduce a correct Friedmann-like equation on the brane, and hence, to overcome the severe cosmo-logical problem in Randall-Sundrum-1 model. Interestingly, it is found that the spacing of Kaluza-Klein mass spectrum in this scenario is very tiny, but the light gravitons cannot be observed individually in colliders because of their sufficiently weak interaction with matter fields on the visible brane.Secondly, based on the new proposed Eddington-inspired Born-Infeld gravity theory, we investigate a thick brane model with a scalar field presenting in the five-dimensional background. A domain wall solution is obtained, and further, we find that at low energy the four-dimensional Einstein gravity is recovered on the brane. Moreover, the stability of gravitational perturbations is ensured in this model. Since the Eddington-inspired Born-Infeld theory is equivalent to Einstein’s general relativity in vacuum, but deviates from it when matter is included, our model presents some new figures comparing to the ones based on general relativity.Thirdly, we consider a de Sitter thick brane model in a five-dimensional Weyl inte-grable geometry, which is a generalization of Riemann geometry. We find a solution of this model via performing a conformal transformation to map the Weylian structure into a familiar Riemannian one. The metric perturbations of the model are discussed. For gravitational perturbation, we get the effective modified Poschl-Teller potential in corre-sponding Schrodinger equation for Kaluza-Klein modes of the graviton. So there exists a mass gap between the unique massless bound mode and continuous massive modes. We also find that the model is stable under the scalar perturbation in the metric. The correction to the Newtonian potential on the brane is investigated.In the final part of this thesis, we consider the dilaton mass in breaking conformal invariance based on a soft-wall model as a concrete application of braneworld in AdS/CFT duality.In AdS/CFT correspondence, a soft-wall geometry is dual to a conformal field theory deformed by a single trace of double trace deformation. Generally, the deformations induce an operator condensate (O), and the dilaton is identified by the fluctuation of the condensate. By numerically and analytically solving the mass eigenfunction of the dilaton, we found that for the single trace deformation, when the scaling dimension of the coupling Δ or the confinement intensity v is large enough, the vacuum stability of the system changes from stable to unstable. For the nearly marginal single trace deformation, there is light dilaton referring to CPR mechanism. But for the nearly marginal double trace deformation, there is not light dilaton, and this is due to a large value of renormalization group flow at the condensate scale. |