The substantial increase in the amount of information over the Internet has contributed to an extraordinary demand for persistent data storage. Centralized storage architectures are expensive, weakly scalable and vulnerable to attacks as they represent single points of failure in the system. Over last few years, peer-to-peer architectures have emerged as an alternative for implementing persistent data-storage. Open peer-to-peer systems are fundamentally scalable and cheaper than client-server approaches. However, in order to successfully build persistent storage systems using the peer-to-peer approach two fundamental challenges need to be addressed. a) To cope with the transient connectivity of peers. b) To reduce the impact of misbehaving peers. Replication is a common approach used to cope with transient connectivity in peer-to-peer storage systems. However, depending on the frequency peers join and leave the system this approach can present negative impacts in terms of storage overhead and bandwidth consumption. Peer-to-peer overlays that focus on tolerating the presence of Byzantine peers usually make the assumption that no more than a bounded fraction of peers in the system are malicious. However, estimating the proportion of malicious peers in open peer-to-peer system is not reliable. Thus, finding a scalable architecture to provide reliable and persistent data storage while coping with these issues is aninteresting achievement. In this thesis we present the design of Datacube. Datacube is an efficient and scalable peer-to-peer storage architecture that provides data persistence by implementing a hybrid redundancy scheme on top of a cluster-based structured overlay. The hybrid redundancy scheme proposed by Datacube ensures data persistence and integrity despite the intermittent connection of peers and the presence of adversarial peers. Datacube relies on the properties of the new class of rateless erasure codes to implement its hybrid redundancy scheme. The analytical evaluations have shown that Datacube performs notably well in terms of availability, storage overhead and bandwidth. Additionally, empirical evaluations have shown the performance of rateless erasure codes in the context of peer-to-peer storage systems. These evaluations helped to understand how the coding parameters impact on the performance of the architecture. To the best of our nowledge, this is the first comprehensive study that helps application designers in finding the values for the coding parameters to best fit their peer-to-peer context.
