Experiences in developing a node of an international computational physics data grid

Abstract

The International Lattice Data Grid (ILDG) is an international collaboration that creates standards to enable sharing of data produced by Lattice Quantum ChromoDynamics (QCD) simulations, which are very computationally expensive. In this paper we summarize our experiences in developing an Australian node of the ILDG, including implementing a program to generate metadata conforming to the QCDml XML schema developed by the ILDG, storing the metadata in an XML database, developing metadata catalog and file catalog services using standard web service interfaces, providing data download using an SRM interface, and implementing a web portal to the ILDG. One of the problems we encountered in this work was that existing metadata catalog software was not able to meet the requirements of this project. Although this work is specific to Lattice QCD data and the ILDG specifications, most of the main issues in developing the data grid node are quite general and not specific to the type of data.