Publication in conference proceedings
Migrating large output applications to Grid environments: a simple library for threaded transfers with gLite
Paulo Abreu (Abreu, P.); Ricardo Fonseca (Fonseca, R. A.); Luís Miguel Oliveira e Silva (Silva, L. O.);
IBERGRID 2008 - The 2nd Iberian Grid Infrastructure Conference
Year (definitive publication)
2008
Language
English
Country
Portugal
More Information
Web of Science®

Times Cited: 0

(Last checked: 2025-12-14 01:46)

View record in Web of Science®

Scopus

This publication is not indexed in Scopus

Google Scholar

Times Cited: 1

(Last checked: 2025-12-11 19:34)

View record in Google Scholar

This publication is not indexed in Overton

Abstract
As the EGEE infrastructure and middleware matures, it is becoming increasingly attractive to different kinds of user applications than it was initially planned for. In particular, it starts to attract users of the application paradigm where, from a relatively small input data set, a large set of data is produced, like it is common in several types of simulations. In this paper we present a simple library based on EGEE middleware function calls, in particular lcg utils, that handles the transfer of large data file sets from the local storage of the Worker Node to an EGEE Storage Element in a way that is simple, robust, transparent to the user and minimizes performance impact. The library is very easy to integrate into non-Grid/legacy code and it is flexible enough to allow for overlapping of several data transfers and computation. We also present a carefull analysis of the performance impact on several application scenarios. Finally, we analyse the use of the library in a real case scenario, where it is used in a high-performance plasma simulation code.
Acknowledgements
--
Keywords
  • Physical Sciences - Natural Sciences
Funding Records
Funding Reference Funding Entity
SFRH/BD/17870/2004 Fundação para a Ciência e a Tecnologia
GRID/GRI/81800/2006 Fundação para a Ciência e a Tecnologia