| The proliferation of commodity cloud services helps developers build wide-area "system-of-systems" applications by harnessing cloud storage, CDNs, and public datasets as reusable building blocks. But to do so, developers must contend with two long-term challenges. First, whenever developers change storage providers, they must work to preserve the application's expected storage semantics, i.e. the rules governing how the application expects the storage provider to handle its reads and writes. Today, changing storage providers is costly, because developers need to patch the application to make it compatible with the new provider's data consistency model, access controls, replica placement strategies, and so on.;At the same time, users have certain expectations about how their data will be used, which the application must meet. For example, depending on the application, users may expect that their data will be kept private from other users, that their data will be exportable to other applications, that accesses to their data will be logged in an auditable way, and so on. In the limit, each user's expectations represent an implicit policy constraining how their data can be stored. Honoring these policies is difficult for developers who rely on third-party storage providers because the storage provider is often unaware of them.;This thesis addresses these challenges with a new wide-area storage paradigm, called "software-defined storage" (SDS), that runs in-between applications and cloud services. SDS-enabled applications do not host data, but instead let users bring their preferred cloud services to the application. By taking a user-centric approach to hosting data, users are empowered to programmatically specify their policies independent of their applications and select services that will honor them. To support this approach and to tolerate service provider changes, SDS empowers developers to programmatically specify their application's storage semantics independent of storage providers.;This thesis presents the design principles for SDS, and validates their real-world applicability with two SDS implementations and several non-trivial applications built on top of them. Most of these applications are used in production today. This thesis presents microbenchmarks of the SDS implementations and uses real-world experiences to show how to make the most of SDS. |
