JOVIAN SCALABLE I/O LIBRARY

Robert Bennett, Kelvin Bryant, Alan Sussman, University of Maryland

There has been a great deal of recent interest in parallel input/output (I/O). The Jovian library is being designed and implemented at the University of Maryland to optimize the I/O performance of multiprocessor architectures that include multiple disks or disk arrays.

High I/O bandwidths can be obtained by declustering files among a number of disks or disk arrays, which are attached to processors. An architecture can be configured so that the disks are attached to processors that have primary responsibility for I/O or the architecture can be configured so that processors provide both computational and I/O support. A key objective of a library designed to optimize parallel I/O is to coalesce disk access requests in a way that makes it possible to present each secondary storage device with a minimal number of disk access requests.

A large fraction of parallelized programs make use of a Single Process Multiple Data (SPMD) model of computation. SPMD programs frequently need to move large sets of data from secondary storage to each processor's primary memory. In the SPMD model, large data sets are distributed across a group of processors in some fashion to achieve parallelism. Also, there is another distribution that must be taken into account when performing I/OÑthe distribution of data over the parallel disk system. More often than not, these distributions are not the same, possibly leading to a great deal of disk head movement. From I/O requests provided by the user, the Jovian library attempts to minimize the number of I/O requests to disk by coalescing them into larger contiguous requests. The library can make use of a varying number of coalescing processes (fixed before program execution) to carry out this aggregation process. This kind of optimization can be called collective local I/O optimization.

Preliminary performance measurements obtained from benchmarking the library on two application templates on the IBM SP1 show that these optimizations can provide significant improvements over traditional I/O methods for such machines.

Table of Contents