Sequential Decision Making in Decentralized Systems

We study sequential decision making problems in cooperative systems where different agents with different information want to achieve a common objective. The sequential nature of the decision problem implies that all decisions can be arranged in a sequence such that the information available to make the tth decision only depends on preceding decisions. Markov decision theory provides tools for addressing sequential decision making problems with classical information structures. In this thesis, we introduce a new approach for decision making problems with non-classical information structures. This approach relics on the idea of common information between decision-makers. Intuitively, common information consists of past observations and decisions that are commonly known to the current and future decision makers. We show that a common information based approach can allow us to discover new structural results of optimal decision strategies and provide a sequential decomposition of the decision-making problems. We first demonstrate this approach on two specific instances of sequential problems, namely, a real-time multi-terminal communication system and a decentralized control system with delayed sharing of information. We then show that the common information methodology applies more generally to any sequential decision making problem. Moreover, we show that our common information methodology unifies the separate sequential decomposition results available for classical and non-classical information structures. We also present sufficient conditions for simplifying common information based sequential decompositions. This simplification relies on the concept of state sufficient for the input output map of a coordinator that only knows the common information.