A Novel Parallel Transmission Strategy for Data Grid

QU Ming-Cheng, WU Xiang-Hu, Yang Xiao-Zong

Abstract


Creation of multi-copies accelerates data transmission and reduces network traffic, but it causes overhead storage and additional network traffic. A variety of parallel transmission algorithms based on GridFTP and multi-copy can be used to accelerate data transmission further, but they can not adapt to a wide range of network, and they can not be used to solve the problems of storage space and network traffic waste. GridTorrent combined with BitTorrent and GridFTP has compatibility with grid and has flexible scalability, but the speed is very slow when there are few peers, to solve this problem multicopy is needed also. To achieve multiple optimization objectives of storage space saving, suitable for two kinds of application modes(i.e. parallel transfer based on GridFTP and BitTorrent), adaptability for wide range of network and higher performance when there are fewer peers, based on the idea of GridTorrent, a distributed storage model, parallel transfer algorithm and virtual peer strategy are proposed. In experiments the performance is compared among the verification system VPG-Torrent and original parallel transfer algorithm
(DCDA) only based on GridfTP & multi-copy and GridTorrent. When the same amount of data is deployed VPG-Torrent has better performance than DCDA, and when there are fewer peers VPG-Torrent also exceed GridTorrent, which prove the effectiveness of VPG-Torrent.


Keywords


Data grid, distributed storage model, parallel transmission

Full Text:

PDF

References


CHEN Lei, LI San-li. A Calking Dynamic Replication Distribution Algorithm in Data Grid. ACTA ELECTRONICA SINICA, 34(11):1-4, 2006

XIE Xiao-lan, LIU Yu, ZHOU De-jian. Research on Manufacturing Grid Data Access and Integration Key Technology. JOURNAL OF WUHAN UNIVERSITY OF TECHNOLOGY, 31(6):1-4, 2009

ZHANG Guangzhi, HE Jieyue. Application Research on Biological Data Grid. Computer Engineering,(2):1-4, 2004

QIN Xin, LUO Ze, NAN Kai etal. Design and Implementation of Problem Solving Environment for Astronomy Application Based on Science Data Grid. Application Research of Computers,(4):1-4, 2009

H.A. James, K.A. Hawick. Scientific Data Management in a Grid Environment. Journal of Grid Computing,3: 39-51, 2005
http://dx.doi.org/10.1007/s10723-005-5464-y

Mingwei Wang, Shusheng Zhang, Jingtao Zhou etal. An Architecture of Semantic Desktop Data Grid. Proceedings of the 10th International Conference on Computer Supported Cooperative Work in Design,IEEE Computer Society, 1-6, 2006
http://dx.doi.org/10.1007/11686699_1

S. Fiore, M. Mirto, Cafaro. A GRelC based Data Grid Management Environment. 21st IEEE International Symposium on Computer-Based Medical Systems, IEEE Computer Society, 355- 360,2008

Richard McClatchey, Ashiq Anjum etal. Data Intensive and Network Aware (DIANA) Grid Scheduling. Journal of Grid Computing,5:43-64, 2007
http://dx.doi.org/10.1007/s10723-006-9059-z

H. Liu, et al., Scheduling jobs on computational grids using a fuzzy particle swarm optimization algorithm, Future Generation Computer Systems:1-8,2009

Xiangang Zhao, BaiWang, Nan Du. Qos-based Algorithm for Job Allocation and Scheduling in Data Grid. Proceedings of the Fifth International Conference on Grid and Cooperative Computing Workshops (GCCW'06), IEEE Computer Society:1-7,2006

Nhan Nguyen Dang, Soonwook Hwang, Sang Boem Lim. Improvement of Data Grid's Performance by Combining Job Scheduling with Dynamic Replication Strategy. The Sixth International Conference on Grid and Cooperative Computing(GCC 2007), IEEE Computer Society:1-8,2007
http://dx.doi.org/10.1109/GCC.2007.79

Esther Pacitti. Patrick Valduriez. Marta Mattoso. Grid Data Management: Open Problems and New Issues. Journal of Grid Computing,5:273-281, 2007
http://dx.doi.org/10.1007/s10723-007-9081-9

Jiang Jianjin, Yang Guangwen. Replication Strategies in Data Grid Systems with Clustered Demands. JOURNAL OF COMPUTER RESEARCH AND DEVELOPMENT,46(2):1-8,2009

W u Chang-ze, Chen Shu-yu, Ti an Dong. The strategy of creating replica based on cost shared in data grid. Huazhong Univ. of Sci. & Tech. (Nature Science Edition),35(2):1-4, 2007

Pangfeng Liu. Jan-Jan Wu, Optimal Replica Placement Strategy for Hierarchical Data Grid Systems. Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid:IEEE Computer Society: 1-4, 2006

Tim Ho, David Abramson. A Unified Data Grid Replication Framework. Proceedings of the Second IEEE International Conference on e-Science and Grid Computing: IEEE Computer Society: 1-8, 2006

Ingmar Baumgart, Bernhard Heep, Stephan Krause, OverSim: A scalable and flexible overlay framework for simulation and real network applications, Proceedings of the 9th International Conference on Peer-to-Peer Computing (IEEE P2P'09 ), pp. 87-88, Seattle, WA, USA, Sep 2009
http://dx.doi.org/10.1109/p2p.2009.5284505

Ingmar Baumgart, Bernhard Heep, Stephan Krause, OverSim: A Flexible Overlay Network Simulation Framework, Proceedings of 10th IEEE Global Internet Symposium (GI '07) in conjunction with IEEE INFOCOM 2007, p. 79-84, Anchorage, AK, USA, May 2007
http://dx.doi.org/10.1109/gi.2007.4301435

R.S.Bhuvaneswaran, Yoshiaki Katayama, Naohisa Takahashi. Dynamic Co-allocation Scheme for Parallel Data Transmission in Grid Environment. Proceedings of the First International Conference on Semantics, Knowledge, and Grid, IEEE Computer Society: 1-6, 2006

Sudharshan, Vazhkudai. Distributed Downloads of Bulk, Replicated Grid Data. Journal of Grid Computing,2:31-42, 2005

Gaurav Khanna, Umit Catalyurek, Tahsin Kurc, et al. A Dynamic Scheduling Approach for Coordinated Wide-Area Data Transfers using GridFTP. The 22nd International Parallel and Distributed Processing Symposium (IPDPS '08). IEEE Computer Society, 2008,1-12

Liu Dongmei, Liu Dongmei. Multi-path parallel transmission scheme for optical grid systems. Chinese High Technology Letters,5:1-4,2008
http://dx.doi.org/10.1016/j.cclet.2007.11.012

A. Zissimos, K. Doka, A. Chazapis and N. Koziris. GridTorrent: Optimizing data transfers in the Grid with collaborative sharing. in Proceedings of the 11th Panhellenic Conference on Informatics (PCI2007), Patras, Greece, May 2007:1-12

Athanasia Asiki, Katerina Doka, Ioannis Konstantinou, et al. A Distributed Architecture for Multi-Dimensional Indexing and Data Retrieval in Grid Environments. In Proceedings of the Cracow 2007 Grid Workshop (CGW'07), Krakow, Polland, October 16-17, 2007:1-8

A. Kaplan, G.C. Fox and G. von Laszewski, GridTorrent Framework: A High-performance Data Transfer and Data Sharing Framework for Scientific Computing. Proc Grid Computing Environments, Supercomputing Workshops, Reno, NV, USA, November 2007:1-10




DOI: https://doi.org/10.15837/ijccc.2011.4.2095



Copyright (c) 2017 QU Ming-Cheng, WU Xiang-Hu, Yang Xiao-Zong

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

CC-BY-NC  License for Website User

Articles published in IJCCC user license are protected by copyright.

Users can access, download, copy, translate the IJCCC articles for non-commercial purposes provided that users, but cannot redistribute, display or adapt:

  • Cite the article using an appropriate bibliographic citation: author(s), article title, journal, volume, issue, page numbers, year of publication, DOI, and the link to the definitive published version on IJCCC website;
  • Maintain the integrity of the IJCCC article;
  • Retain the copyright notices and links to these terms and conditions so it is clear to other users what can and what cannot be done with the  article;
  • Ensure that, for any content in the IJCCC article that is identified as belonging to a third party, any re-use complies with the copyright policies of that third party;
  • Any translations must prominently display the statement: "This is an unofficial translation of an article that appeared in IJCCC. Agora University  has not endorsed this translation."

This is a non commercial license where the use of published articles for commercial purposes is forbiden. 

Commercial purposes include: 

  • Copying or downloading IJCCC articles, or linking to such postings, for further redistribution, sale or licensing, for a fee;
  • Copying, downloading or posting by a site or service that incorporates advertising with such content;
  • The inclusion or incorporation of article content in other works or services (other than normal quotations with an appropriate citation) that is then available for sale or licensing, for a fee;
  • Use of IJCCC articles or article content (other than normal quotations with appropriate citation) by for-profit organizations for promotional purposes, whether for a fee or otherwise;
  • Use for the purposes of monetary reward by means of sale, resale, license, loan, transfer or other form of commercial exploitation;

    The licensor cannot revoke these freedoms as long as you follow the license terms.

[End of CC-BY-NC  License for Website User]


INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL (IJCCC), With Emphasis on the Integration of Three Technologies (C & C & C),  ISSN 1841-9836.

IJCCC was founded in 2006,  at Agora University, by  Ioan DZITAC (Editor-in-Chief),  Florin Gheorghe FILIP (Editor-in-Chief), and  Misu-Jan MANOLESCU (Managing Editor).

Ethics: This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE).

Ioan  DZITAC (Editor-in-Chief) at COPE European Seminar, Bruxelles, 2015:

IJCCC is covered/indexed/abstracted in Science Citation Index Expanded (since vol.1(S),  2006); JCR2018: IF=1.585..

IJCCC is indexed in Scopus from 2008 (CiteScore2018 = 1.56):

Nomination by Elsevier for Journal Excellence Award Romania 2015 (SNIP2014 = 1.029): Elsevier/ Scopus

IJCCC was nominated by Elsevier for Journal Excellence Award - "Scopus Awards Romania 2015" (SNIP2014 = 1.029).

IJCCC is in Top 3 of 157 Romanian journals indexed by Scopus (in all fields) and No.1 in Computer Science field by Elsevier/ Scopus.

 

 Impact Factor in JCR2018 (Clarivate Analytics/SCI Expanded/ISI Web of Science): IF=1.585 (Q3). Scopus: CiteScore2018=1.56 (Q2); Editors-in-Chief: Ioan DZITAC & Florin Gheorghe FILIP.