TY - GEN
T1 - Optimal Code Regeneration with Background Traffic Awareness in Distributed Storage
AU - Tao, Yangyang
AU - Yu, Shucheng
AU - Yoshigoe, Kenji
AU - Zhou, Junxiu
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/6/19
Y1 - 2018/6/19
N2 - In cloud storage systems a certain degree of data redundancy is important for data availability. Timely regeneration of corrupted or lost data shares is desired to meet the MTTR (mean time to recovery) reliability requirements as usually defined in Service Level Agreements (SLA). Current data regeneration techniques usually assume uniform and/or unlimited network capacity while ignoring the impacts of background traffics and cloud network architecture in practice. This paper proposes a more realistic regeneration strategy by taking these impacts into consideration. Specifically, our approach first extracts an information flow graph from BCube network architecture based on which the real-time network status is predicted using a Markov Chain model. The optimal code regeneration strategy is then formulated as a linear programming (LP) problem which minimizes the sub-flow rate on bottleneck links subject to the constraint of real-time network dynamics. Finally, a distributed multi-commodity flow dynamic routing (MFDR) approximation algorithm is proposed to solve the code regeneration LP. Simulation results indicate that the proposed distributed algorithm on average saves 16.5% data regeneration time of RCTREE and 45.3% of HDFS.
AB - In cloud storage systems a certain degree of data redundancy is important for data availability. Timely regeneration of corrupted or lost data shares is desired to meet the MTTR (mean time to recovery) reliability requirements as usually defined in Service Level Agreements (SLA). Current data regeneration techniques usually assume uniform and/or unlimited network capacity while ignoring the impacts of background traffics and cloud network architecture in practice. This paper proposes a more realistic regeneration strategy by taking these impacts into consideration. Specifically, our approach first extracts an information flow graph from BCube network architecture based on which the real-time network status is predicted using a Markov Chain model. The optimal code regeneration strategy is then formulated as a linear programming (LP) problem which minimizes the sub-flow rate on bottleneck links subject to the constraint of real-time network dynamics. Finally, a distributed multi-commodity flow dynamic routing (MFDR) approximation algorithm is proposed to solve the code regeneration LP. Simulation results indicate that the proposed distributed algorithm on average saves 16.5% data regeneration time of RCTREE and 45.3% of HDFS.
KW - Cloud Storage
KW - Data Regeneration
KW - Linear Programming
KW - Markov Chain
KW - Multi-commodity Flow
KW - SRB-X
UR - http://www.scopus.com/inward/record.url?scp=85050139036&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050139036&partnerID=8YFLogxK
U2 - 10.1109/ICCNC.2018.8390329
DO - 10.1109/ICCNC.2018.8390329
M3 - Conference contribution
AN - SCOPUS:85050139036
T3 - 2018 International Conference on Computing, Networking and Communications, ICNC 2018
SP - 48
EP - 52
BT - 2018 International Conference on Computing, Networking and Communications, ICNC 2018
T2 - 2018 International Conference on Computing, Networking and Communications, ICNC 2018
Y2 - 5 March 2018 through 8 March 2018
ER -