Data futures in DISCWorld

Abstract

Data futures in a metacomputing system refer to data products that have not yet been created but which can be uniquely named and manipulated. We employ data flow mechanisms expressed as high level task graphs in our DISCWorld metacomputing system. Nodes in these task graphs can themselves be expanded into further task graphs or can be represented as futures in the processing schedule. We find this this a generally useful approach to the dynamic and optimal execution of task graphs. In this paper we describe our DISCWorld Remote Access Mechanism for Futures (DRAMFs) and its implementation using Java technology. DRAMFs embody these ideas and allow data products to be lazily de-referenced and to be manipulated as first class objects in the object space of the metacomputing system. Our system aids in the provision against partial failure or network disruptions in a distributed system such as DISCWorld.