The genetic basis of reproductive isolation between two species of house mice

Determining the genetic basis of reproductive isolation is a fundamental goal in evolutionary biology. Intrinsic reproductive isolation often arises due to epistasis between divergent interacting genes. The rapid evolution of hybrid male sterility is known to have several causes, including the exposure of recessive X-linked incompatibilities in males and the rapid evolution of male reproductive traits. Despite these insights, little is known about the genetics of reproductive isolation during the early stages of speciation. This deficiency inspired parallel studies on the molecular evolution of male reproduction in house mice and the genetic basis of hybrid male sterility between two mouse species, Mus domesticus and M. musculus. Evolutionary analysis of 946 genes showed that the intensity of positive selection varies across sperm development and acts primarily on phenotypes that develop late in spermatogenesis (Appendix A). Several reciprocal crosses between wild-derived strains of M. musculus and M. domesticus were used to examine F1 hybrid male sterility (Appendix B). These crosses revealed hybrid male sterility linked to the M. musculus X chromosome and a novel sterility polymorphism within M. musculus. A large introgression experiment was used to further dissect the genetic basis of X-linked incompatibilities between M. musculus and M. domesticus (Appendix C). Introgression of the M. musculus X chromosome into a M. domesticus genetic background produced male sterility and involved a minimum of four factors. No sterility factors were uncovered on the M. domesticus X chromosome. These data demonstrate the complex genetic basis of hybrid sterility in mice and provide numerous X-linked candidate sterility genes. The molecular evolution of five rapidly evolving candidate genes was examined using population and phylogenetic sampling in Mus (Appendix D). Four of these loci showed evidence of positive natural selection. One locus, 4933436I01Rik, showed divergent protein evolution between M. domesticus and M. musculus and was one of a handful of testis-expressed genes within a narrow interval involved in hybrid male sterility. In summary, these data demonstrate that hybrid male sterility has a complex genetic basis between two closely related species of house mice and provide a foundation for the identification of specific mutations that isolate these species.