Distributed Gridded Data Delivery for Marine Research

Abstract

A combination of off-the-shelf open-source software and custom-built middleware is used to unite remotely sensed marine data archives operated across Australia by five different agencies and make them accessible via a common interface to a user located anywhere on the internet. The utilisation of existing storage and some state of the art fileservers with a distributed data model, makes the system low-cost, scalable and robust. The creation of a virtual national data set with automatic cataloguing enables the development of advanced data services including aggregation and spatio-temporal time series and sub-setting. Data sets are served through OPeNDAP and automatically harvested for meta-data including temporal and spatial bounds. Spatio-temporal queries made on the catalogues provide information to allow retrieval of subset data, or if many data sets are returned, the aggregation of data matching the query. The software developed to implement the system is built as a set of layers with well defined interfaces, allowing system modularisation and a range of levels of access.