RAMCloud: The Future of Storage Systems
In RAMCloud, data is stored in the DRAM of thousands of computers in a data centre. RAMCloud offers quick and reliable recovery even though terabytes of data may be stored in the system.
Today, the amount of data generated on the Internet is enormous. An application like Facebook needs to deal with petabytes or terabytes of data without compromising its performance. RAMCloud is a next generation storage system that can deliver high performance with the help of just commodity hardware, even at current levels of storage complexity. It stores data entirely in DRAM (main memory), and the disk takes the role of backup or archival storage. Since the data always resides in the main memory, it can avoid the access latency that is usually incurred in a disk-based storage system and, hence, provides high throughput, which is the key to better performance.
The current scenario
For the past four decades we have seen rapid growth in computer hardware technologies, which has helped improve the efficiency of storage systems. The processor, memory and disk play an important role in the performance of a storage system. Currently, an imbalance in the performance of any one of these components can impact the whole system. Though there has been tremendous improvement in the performance of both memory and processor, the disk has not been able to keep pace. Disk capacity has increased by more than a thousand times, but the transfer rate for large blocks has improved only fifty-fold, while seek time and rotational latency have only improved two-fold.
Large applications like Facebook and Amazon require multiple access points to storage servers to generate a single page. Due to the high access rate the performance of these applications is reduced. Applications use cache to overcome disk latency, but cache must have an exceptionally high hit rate to provide significant performance improvement. Even a 1 per cent cache miss can severely affect the system’s performance, which is not acceptable for some applications.
Flash memory is another storage system that offers latency lower than disk. But Flash devices are I/O devices, so apart from the access latency, they have additional latencies of device drivers and interrupt handlers. These shortcomings of current storage systems demand a new improved storage approach.
An overview of RAMCloud
RAMCloud is a storage system that stores data in the DRAM of thousands of servers within a data centre, as shown in Figure 1.
Since the information is kept in the DRAM at all times, the access rate is very low, and provides 100-1000x lower latency than disk-based systems and 100-1000x greater throughput. Most Web applications grow over a period of time and will require more servers to store their data. RAMCloud will scale automatically to support the growing number of servers added to the system.
RAMCloud uses DRAM, which has volatile memory, i.e., the data is lost when the power is removed. However, applications require storage systems to provide a high level of data durability and availability. RAMCloud uses a technique called buffered logging to maintain durability. In this approach, a single copy of each data object is stored in the DRAM of a primary server on the disks of two or more backup servers; each acts as both primary and backup server. When a write operation is performed, the primary server updates its DRAM and forwards log entries to the backup servers, where they are stored temporarily in the DRAM of the backup server. The backup server collects log entries into batches that can be written efficiently to a log on disk. Once log entries have been written to disk they can be removed from the backup's DRAM.
Buffered logging allows both reads and writes to proceed at DRAM speeds while still providing durability. Power failures can be handled by committing each write operation to a stable storage.
The data model
The data model for a storage system governs how data is collected, stored, integrated and put to use. There are three main factors that we need to decide on, prior to selecting the type of data model: The nature of the basic objects stored in the system. How basic objects are organised into higher-level structures; for example, we can either just have key-value pairs or some sort of aggregation. Select the methods for naming and indexing of objects when retrieving or modifying objects. The two common types of data models are the highly structured relational data model and the unstructured data model. RAMCloud prefers an intermediate approach where servers do not impose structure on data but do support aggregation and indexing. It supports any number of tables, each stores multiple objects and these objects are stored as simple key-value pairs. It also provides a simple set of operations for creating and deleting tables, and for reading, writing and deleting objects within a table.
Research challenges
Numerous challenges need to be fixed for RAMCloud to be implemented successfully. Given below is a short description of various challenges that researchers are trying to solve.
Consider the applications that use TCP/IP protocol - they have round-trip times for remote procedure calls and high latency in network switches. Also, the flow oriented feature of TCP is of little use to RAMCloud, since individual requests will be relatively small. To improve overall latency, we can either modify or replace the TCP protocol with a UDP protocol. An increasing number of applications are using virtualisation for greater portability. This increases the overheads, since an incoming packet will now need to pass through the virtual machine monitor and a guest operating system before reaching the application, thereby increasing the overall latency. So we need techniques like passing packets directly from the virtual monitor to the application to reduce this overhead.
RAMCloud is implemented by using a large number of servers, but the applications that use them, must see this as a single storage system, i.e., the application must be oblivious to the distribution of the storage system. The primary issue in the distribution and scaling of the system is data placement. An object may be required to be moved to another server to improve the performance. This data movement needs to
happen automatically and in real time.
A single RAMCloud system can be used to support multiple applications of varying sizes. It should provide a security mechanism to support mutual hostile applications. Also, one application with a very high workload must not degrade the performance of other applications.
Finally, RAMClouds must manage themselves automatically. There are thousands of servers, each using hundreds of peers, which makes the overall design too complicated to be handled by humans.
Why use RAMCloud?
We believe there are two main motivations for using RAMCloud.
Application scalability
Most Web applications use relational databases to store their data. As the application grows, it becomes difficult to store the entire data in a single relational database. Applications then use other techniques to manage their data. A popular technique is ‘Ad-hoc’, where data is partitioned among multiple databases. As the application grows larger, maintaining consistency among multiple databases becomes increasingly difficult and requires more complex techniques to overcome these issues. Another storage technique is ‘Bigtable’, which is built on top of a Google file system. Because of the distributed nature of a Bigtable database, performing certain database operations like a join between two tables would be terribly inefficient. On the other hand, RAMCloud will automatically scale to support the increasing number of storage servers used by an application.
The technology
The disk is used as a primary storage system for Web applications. Accessing large blocks at one time from a disk may be beneficial when compared to accessing small blocks. However, most forms of online data, such as images and songs, do not comprise large blocks. So the latency for accessing the smaller more frequent blocks of memory is high. Large Web applications need to make multiple internal requests to generate a single HTML page. So we need to consider the cumulative latency of all the requests while considering the overall response time to users. One of the major advantages of RAMCloud over the diskoriented approach is that it can dramatically reduce the access latency of a request and, thereby, reduce the overall response time. RAMCloud also supports a new class of data-intensive applications, which process data in large volumes—typically, in terabytes.
The pros and cons
Here are some of the pros and cons of using RAMCloud.
The pros
1. Since all the information is stored in DRAM, RAMCloud
provides high throughput.
2. RAMCloud automatically scales to support a large number of storage servers and eliminates the scalability issues in applications.
3. It provides high level of data durability and availability.
4. The cost of storing data on DRAM today is the same as storing data on a disk ten years ago.
5. RAMCloud supports a log-structure similar to a log structured file system for all its data on DRAM as well as on disk. This provides fast crash recovery. 6. RAMClouds are 100-1000x more efficient than disk-based systems in terms of cost per operation or energy per operation.
The cons
1. It involves a higher cost/bit and high energy/bit, so RAMCloud storage will be 50-100x more expensive than a pure disk-based system.
2. Maintaining consistency for applications that require replication across data centres is very difficult. With the growth of large scale Web applications, there has been a need for alternative disk storage technologies. Both Google and Yahoo store their search indices entirely in DRAM. The Bigtable storage system allows entire column families to be loaded into memory, where they can be read without any disk accesses. We believe that RAMCloud is a long-term solution for the storage needs of Web applications. RAMCloud provides durability and very low latency. Hence, it enables richer query models and is attractive for technologies like cloud computing. It is able to aggregate the resources of a large number of commodity servers. However, a lot of research needs to be done and numerous challenges must be overcome in order to use this technology.
References
[1] https://ramcloud.stanford.edu [2] http://www.stanford.edu/~ouster/cgi-bin/papers/ramcloud.pdf
Acknowledgements
I would like to thank Dr John Ousterhout, professor of Computer Science at Stanford University. He is the lead at the RAMCloud project at Stanford University. I would also like to thank my mentor and all the people who helped me to review this article.