Ceph and the SSRC

Ceph came out of the Storage Systems Research Center (SSRC) at UC Santa Cruz, which Dr. Ethan Miller and I built together. I coauthored the 2006 Ceph paper with Dr. Sage Weil, Dr. Scott Brandt, Dr. Ethan Miller, and Dr. Carlos Maltzahn.

The research was sponsored as part of the Department of Energy’s Accelerated Strategic Computing Initiative (ASCI), through Gary Grider at Los Alamos National Laboratory (LANL), Steve Louis at Lawrence Livermore National Laboratory (LLNL), and Lee Ward at Sandia National Laboratories. Their support made this work possible.

Swift and RAMA

The separation of metadata from data, with separate servers handling each, came from my work with Dr. Luis-Felipe Cabrera on Swift, beginning in 1989. A client obtained the information it needed from the metadata service and then transferred data directly to and from the storage servers. The metadata server did not have to carry the data traffic.

We presented this work at the 1991 Summer USENIX conference. Our Swift paper in Computing Systems, also published in 1991, describes the architecture. The same separation later appeared in IBM’s Storage Tank, which became IBM TotalStorage SAN File System.

Another part of Ceph’s history is Dr. Miller’s Ph.D. work at Berkeley on RAMA. RAMA used hashing to distribute data across storage devices. Dr. Miller and his collaborators developed that line of research into RUSH and CRUSH. CRUSH lets clients calculate where data belongs without consulting a central allocation directory, while accounting for replication, changes in the storage cluster, and the need to keep replicas in separate failure domains.

Dr. Weil’s Work

Dr. Weil did brilliant work of his own in his dissertation, including scalable metadata management and the CRUSH algorithm. He then led the work of turning Ceph from a research file system into a storage system for production use. After his doctorate, he continued development with a team at DreamHost and cofounded Inktank to develop and support Ceph commercially. Red Hat acquired Inktank in 2014.

Ceph is now used in some of the world’s largest research and cloud storage installations. Dr. Weil followed his research with years of engineering and led the developers and company that supported Ceph’s adoption.