Dr. Darrell Long

Geomancy: Automated Performance Enhancement through Data Layout Optimization

Proceedings of the Conference on Mass Storage Systems and Technologies (MSST)

October 2020View full paper

Abstract

Large distributed storage systems such as high-performance computing (HPC) systems used by national or international laboratories require sufficient performance and scale for demanding scientific workloads and must handle shifting workloads with ease. Ideally, data is placed in locations to optimize performance, but the size and complexity of large storage systems inhibit rapid effective restructuring of data layouts to maintain performance as workloads shift.

To address these issues, we have developed Geomancy, a tool that models the placement of data within a distributed storage system and reacts to drops in performance. Using a combination of machine learning techniques suitable for temporal modeling, Geomancy determines when and where a bottleneck may happen due to changing workloads and suggests changes in the layout that mitigate or prevent them. Our approach to optimizing throughput offers benefits for storage systems such as avoiding potential bottlenecks and increasing overall I/O throughput from 11% to 30%.

BibTeX

@inproceedings{bel-msst20,
  author       = {Oceane Bel and Kenneth Chang and Nathan R. Tallent and Dirk Duellmann and Ethan L. Miller and Faisal Nawab and Darrell D. E. Long},
  title        = {Geomancy: Automated Performance Enhancement through Data Layout Optimization},
  booktitle    = {Proceedings of the Conference on Mass Storage Systems and Technologies (MSST)},
  organization = {IEEE},
  address      = {Santa Clara University},
  month        = oct,
  year         = {2020}
}
View full paper