Optimal Code Regeneration with Background Traffic Awareness in Distributed Storage

Yangyang Tao, Shucheng Yu, Kenji Yoshigoe, Junxiu Zhou

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

In cloud storage systems a certain degree of data redundancy is important for data availability. Timely regeneration of corrupted or lost data shares is desired to meet the MTTR (mean time to recovery) reliability requirements as usually defined in Service Level Agreements (SLA). Current data regeneration techniques usually assume uniform and/or unlimited network capacity while ignoring the impacts of background traffics and cloud network architecture in practice. This paper proposes a more realistic regeneration strategy by taking these impacts into consideration. Specifically, our approach first extracts an information flow graph from BCube network architecture based on which the real-time network status is predicted using a Markov Chain model. The optimal code regeneration strategy is then formulated as a linear programming (LP) problem which minimizes the sub-flow rate on bottleneck links subject to the constraint of real-time network dynamics. Finally, a distributed multi-commodity flow dynamic routing (MFDR) approximation algorithm is proposed to solve the code regeneration LP. Simulation results indicate that the proposed distributed algorithm on average saves 16.5% data regeneration time of RCTREE and 45.3% of HDFS.

Original languageEnglish
Title of host publication2018 International Conference on Computing, Networking and Communications, ICNC 2018
Pages48-52
Number of pages5
ISBN (Electronic)9781538636527
DOIs
StatePublished - 19 Jun 2018
Event2018 International Conference on Computing, Networking and Communications, ICNC 2018 - Maui, United States
Duration: 5 Mar 20188 Mar 2018

Publication series

Name2018 International Conference on Computing, Networking and Communications, ICNC 2018

Conference

Conference2018 International Conference on Computing, Networking and Communications, ICNC 2018
Country/TerritoryUnited States
CityMaui
Period5/03/188/03/18

Keywords

  • Cloud Storage
  • Data Regeneration
  • Linear Programming
  • Markov Chain
  • Multi-commodity Flow
  • SRB-X

Fingerprint

Dive into the research topics of 'Optimal Code Regeneration with Background Traffic Awareness in Distributed Storage'. Together they form a unique fingerprint.

Cite this