6.0 File library interface

The Legion file interface library is a set of C functions that allow Legion file objects to be manipulated from user programs with a familiar interface similar to that provided by the Unix file system. To use the library, programmers should include the header file Legion_libBasicFiles.h and link their programs against the library libBasicFiles.a (located in the same directory as the basic Legion libraries). Linkage against the basic Legion libraries is also required.

A sample Legion File Library makefile is below.

Figure 14Sample Legion File Library makefile
CC       = g++
CFLAGS   = -I$(LEGION)/include -L$(LEGION)/lib/$(LEGION_ARCH)/gnu

include $(LEGION)/src/LibraryCode/macros/$(LEGION_ARCH).$(CC).macros

LIB      = -lBasicFiles -lLegion1 -lLegion2 $(LFLAGS)
 
example: example.c
        $(CC) $(CFLAGS) example.c -o example $(LIB)

This directory contains the Legion basic file objects and libraries for using these objects. Legion basic file objects are quite simple at their core, representing a random-access linear array of bytes.This linear byte array is indexed starting at zero and preceding to (fileSize - s1). Files grow to accommodate the data that are written into them.The random access nature of the basic file object requires clients of the file to maintain their own notion of a file pointer if sequential access is desired. While automatic maintenance of file pointers is not performed by file objects, utility libraries to provide this service are available and are described in Library functions.

There is a set of public methods exported by Legion basic files (see Public methods), but the public interface is somewhat inconvenient in cases where sequential access, and/or frequent, fine-grained access is desirable. To address these issues, a set of library functions is provided to support convenient access to file objects. The interface functions provided with the Legion file library fall into several main categories:

6.1 Public methods

long
write(long startAtByte, BasicFileTransferBlock writeData)

BasicFileTransferBlock
read(long startAtByte, long numBytesToRead)

BasicFileStatBuffer
stat()

int
trunc()

long
append(BasicFileTransferBlock writeData)

long
truncAppend(BasicFileTransferBlock writeData)

6.2 Library functions
6.2.1 Raw I/O library

void
BasicFiles_init

void
BasicFiles_done()

int
BasicFiles_exists(char *path)

BasicFiles_FileDescriptor
BasicFiles_creat(char *path)

BasicFiles_FileDescriptor
BasicFiles_open(char *path, int flags)

int
BasicFiles_close(BasicFiles_FileDescriptor fd)

void
BasicFiles_delete(char *path)

BasicFiles_FileOffset
BasicFiles_read(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)

BasicFiles_FileOffset
BasicFiles_write(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)

BasicFiles_FileOffset
BasicFiles_seek(BasicFiles_FileDescriptor fd,
BasicFiles_Whence whence,
BasicFiles_FileOffset offset)

BasicFiles_FileOffset
BasicFiles_tell(BasicFiles_FileDescriptor fd)

BasicFiles_FileOffset
BasicFiles_size(BasicFiles_FileDescriptor fd)

BasicFiles_FileOffset
BasicFiles_trunc(BasicFiles_FileDescriptor fd)

BasicFiles_FileOffset
BasicFiles_append(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)

BasicFiles_FileOffset
BasicFiles_truncAppend(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)

6.2.2 Buffered I/O library, C and C++ interface

BasicFILE *
BasicFiles_fopen(char *path, char *flags)

int
BasicFiles_setbufsize(BasicFILE *file, int sz)

int
BasicFiles_fclose(BasicFILE *file)

int
BasicFiles_fflush(BasicFILE *file)

BasicFiles_FileOffset
BasicFiles_fseek(BasicFILE *file, BasicFiles_FileOffset offset,
BasicFiles_Whence whence)

BasicFiles_FileOffset
BasicFiles_ftell(BasicFILE *file)

int
BasicFiles_fputc(int c, BasicFILE *file)

int
BasicFiles_fputs(char *str, BasicFILE *file)

int
BasicFiles_fwrite(void *ptr, int size, int nmemb, BasicFILE *file)

int
BasicFiles_fgetc(BasicFILE *file)

char *
BasicFiles_fgets(char *str, int sz, BasicFILE *file)

int
BasicFiles_fread(void *ptr, int size, int nmemb, BasicFILE *file)

6.2.3 Buffered I/O library, Fortran interface

subroutine liof_open(path, flags, fd)
character path(*)
integer flags, fd

subroutine liof_close(fd, ret)
integer flags, ret

subroutine liof_write(fd, buf, sz, ret)
integer fd
character buf(*)
integer sz, ret

subroutine liof_write_line(fd, line, sz, ret)
integer fd
character line(*)
integer sz, ret

	character msg*256
10  format(I8,' .... ',I5)
	write(msg,10) X, Y
	call lio_write_line(fd, msg, 256, ret)

subroutine liof_flush(fd, ret)
integer fd, ret

subroutine liof_read(fd, buf, sz, ret)
integer fd
character buf(*)
integer sz, ret

subroutine liof_read_line(fd, line, sz, ret)
integer fd
character line(*)
integer sz, ret

	character msg*256
10  format(I8,' .... ',I5)
	call lio_read_line(fd, msg, 256, ret)
	read(msg,10) X, Y

subroutine liof_rewind(fd, ret)
integer fd, ret

subroutine liof_write_ints(fd, x, num, ret)
integer fd
integer x(*)
integer num, ret

subroutine liof_read_ints(fd, x, num, ret)
integer fd
integer x(*)
integer num, ret

subroutine liof_write_reals(fd, x, num, ret)
integer fd
real x(*)
integer num, ret

subroutine liof_read_reals(fd, x, num, ret)
integer fd
real x(*)
integer num, ret

subroutine liof_write_doubles(fd, x, num, ret)
integer fd
double precision x(*)
integer num, ret

subroutine liof_read_doubles(fd, x, num, ret)
integer fd
double precision x(*)
integer num, ret

6.2.4 Buffered I/O library, low impact interface

lio_legion_to_tempfile(character*(*) legion_name,
character*(*) tempfile, integer ierr)

lio_create_tempfile(character*(*) tempfile, integer ierr)

lio_tempfile_to_legion(character*(*) tempfile,
character*(*)legion_name, integer ierr)

6.2.5 The LegionBuffer I/O interface

Using the new LegionBuffer type is simple:

UVaL_Reference<LegionBuffer> BasicFiles_openBuffer(char *path);

This function returns a LegionBuffer that can be used to access the file object referred to by path. If no such file exists, it is created.

The standard LegionBuffer interface is described in Basic module and data container: the Legion buffer.

6.2.6 Two-dimensional FileObject interfaces

Parallel applications often encounter a bottleneck while trying to perform file operations [27] [28]. Legion offers a two-dimensional file object, which attempts to address this issue in two ways: it block partitions the file into several subfiles [18] and it allows the application to specify how it wants to access the file so that only needed data are transferred to each of the application's nodes. This makes it easier for the programmer to perform non-sequential I/O operations. And, since the file's decomposition is specified at creation time, the file can be broken up in such a manner as to minimize both disk and network contention within the application during I/O operations. The TwoDFileObject reduces the amount of application code that is required by the standard file interface to perform the same operations, since multiple reads or writes can be bundled into a single request without the need for seeks in-between.

The two-dimensional file object provides users with the following features:

The two-dimensional file object is layered on top of Legion. This means that it can be run on multiple hosts and architectures without any modifications to the underlying file systems. This is particularly useful in cluster computing environments where large files typically come from an NFS file server. Two-dimensional file objects take advantage of Legion's data-coercion mechanism to provide heterogeneous access to the file. Thus the subfiles and the nodes of the application can all be on different architectures but still generate a file with a single data format.

The two-dimensional file object provides five access patterns for both reading and writing. These patterns are access by:

Row and column access patterns allow the program to access several rows or columns of the file at a time. The rows and columns can be separated by a fixed stride. Thus with a single read request the program could gather five columns of the file where each column is separated by ten unneeded columns. The block access pattern allows the program to access a rectangular subsection of the file. This should be particularly useful for stencil problems, which tend to decompose the problem space into regular subsections. The element access pattern allows programs to access a regular pattern of elements within some subsection of the file. This pattern can also be used to access subrows or subcolumns of the file. The sequential access mode allows the file to be viewed as if it were a sequential file. This is intended for pre-existing filters that will access the entire file either before or after the application has run (such as visualization tools). For a description of the calling conventions and usage of these access patterns see Synchronous interface and Asynchronous interface.

Applications use the lib2Dfiles library to access a TwoDFileObject. The two-dimensional file is implemented as a collection of BasicFileObjects (sequential files in Legion) which are controlled by a single master object. To prevent bottlenecks, a library running on the client gets relevant file information from the TwoDFileObject and then communicates directly with the BasicFileObjects. This allows several nodes to access different parts of the file concurrently without contention.

When the library receives a request from an application it iterates over the subfiles and determines if it needs to communicate with that particular subfile. It then builds a list of offsets and lengths within that subfile that are required to satisfy the read request. Next, it sends the list of offsets to the subfile and determines what is required from the remaining subfiles. Once all the requests have been sent to the appropriate subfiles the library iterates over them a second time to gather the results. When the library gets back the results it coerces the data into the correct representation and then places it at the appropriate locations in the result array. Note that only one message is sent to each subfile involved in the request and that the results come back in a single message. This uses the minimum possible amount of communication that can still satisfy the request.

6.2.6.1 Examples

This section contains four sample two-dimensional file object programs. The first three show C++ applications that use various calls to create and use blocks, columns, and rows. The last shows a Fortran application.

The first, Example A, shows a C application that creates a 11x9 2D file consisting of six subfiles. The biggest and smallest subfiles do not differ by more than one row and one column, so that the biggest subfile is 4x5 and the smallest is 3x4. After the 2D file is created it is filled in with a writeBlock call. Finally, the program reads back a sub-block of the file with a readBlock call.

Example A C 2D file program using readBlock and writeBlock calls

#include<stdio.h>
#include<legion/libc_TwoDFileObject.h>

int main()
{
	int tst;
	int i, j, rv;
	int foo[9][11];
	int bar[6][7];

	/* initialize the legion libraries */
	BasicFiles_init();
	fprintf(stderr, "calling create\n");
	/* create the TwoDFileObject and all of its subfiles */
	rv = lio_2d_create_int(&tst, "TwoDFileTestFile", 11, 
		9, 3, 2, NULL);
	fprintf(stderr, "back from create\n");
	/* check return code */
	if(rv < 0)
	{
		fprintf(stderr, "unable to create TwoDFileObject\n");
		return 1;
	}
	/* initialize the foo array */
	for(i = 0; i < 9; i++)
	{
		for(j = 0; j < 11; j++)
		{
			foo[i][j] = (i * 11) + j;
		}
	}
	fprintf(stderr, "calling writeRows\n");
	/* populate the TwoDFileObject */
	rv = lio_2d_writeBlock_int(tst, &foo[0][0], 0, 0, 9,
		11, 11, 9, 0, 0);
	fprintf(stderr, "back from writeRows\n");
	/* check return code */
	if(rv < 0)
	{
		fprintf(stderr, "unable to write to TwoDFileObject\n");
		return 1;
	}
	printf(stderr, "calling readRows\n");
	/* read back a sub-section of the TwoDFileObject */
	rv = lio_2d_readBlock_int(tst, &bar[0][0], 0, 0, 6,
 		7, 7, 6, 0, 0);
	fprintf(stderr, "back from readRows\n");
	/* check return code */
	if(rv < 0)
	{
		fprintf(stderr, "unable to read from TwoDFileObject\n");
		return 1;
	}
	/* print out the results of the read */
	for(i = 0; i < 6; i++)
	{
		for(j = 0; j < 7; j++)
		{
			fprintf(stderr, "%d, ", bar[i][j]);
		}
		fprintf(stderr, "\n");
	}

	return 0;
}

Example B shows a C++ application that reads a 11x9 two-dimensional file consisting of six subfiles. The biggest and smallest subfile do not differ by more than one row and one column, so that the biggest subfile is 4x5 and the smallest is 3x4. After reading the two-dimensional file with readCol, the program deletes the file object.

Example B C++ 2D file program using a readCol call

#include<stdio.h>
#include<legion/lib_TwoDFileObject.h>

main()
{
	int c;
	int i, j, rv;

	// initialize the legion libraries
	BasicFiles_init();

	TwoDFileObject<int> tst;
	LegionArray<int> *bar;

	fprintf(stderr, "calling open\n");
	// open the TwoDFileObject
	rv = tst.open("TwoDFileTestFile");
	fprintf(stderr, "back from open\n");
	// check return value
	if(rv < 0)
	{
		fprintf(stderr, "unable to open TwoDFileObject\n");
		return 1;
	}
	fprintf(stderr, "calling readCols\n");
	// read three columns, skipping over two columns 
	// between each column read
	bar = tst.readCols(1, 3, 3);
	fprintf(stderr, "back from readCols\n");
	// check return value
	if(bar == NULL)
	{
		fprintf(stderr, "unable to read from TwoDFileObject\n");
		return 1;
	}
	// print results of read
	for(i = 0; i < 9; i++)
	{
		for(j = 0; j < 3; j++)
		{
			c = bar->GetElement(i, j);
			fprintf(stderr, "%d, ", c);
		}
	fprintf(stderr, "\n");
	}
	fprintf(stderr, "calling destroy\n");


	// delete the TwoDFileObject	
	rv = tst.destroy();
	fprintf(stderr, "back from destroy\n");

	return rv;
}

Example C shows a C++ application that creates a 11x9 two-dimensional file consisting of six subfiles. The biggest and smallest subfile do not differ by more than one row and one column, so the biggest subfile is 4x5 and the smallest is 3x4. After creating a two-dimensional file, the program fills it in with a writeBlock call. Finally, the program reads back a sub-block of the file with readRow calls.

Example C C++ 2D file program using readRow and writeRow calls

#include<stdio.h>
#include<legion/lib_TwoDFileObject.h>

main()
{
	int c;
	int i, j, rv;

	// initialize the legion libraries
	BasicFiles_init();

	TwoDFileObject<int> tst;
	LegionArray<int> foo(9, 11);
	LegionArray<int> *bar;

	fprintf(stderr, "calling create\n");
	// create the TwoDFileObject and all of its subfiles
	rv = tst.create("TwoDFileTestFile", 11, 9, 3, 2);
	fprintf(stderr, "back from create\n");
	// check return value
	if(rv < 0)
	{
		fprintf(stderr, "unable to create TwoDFileObject\n");
		return 1;
	}
	// initialize the array 
	for(i = 0; i < 9; i++)
	{
		for(j = 0; j < 11; j++)
		{
			c = (i * 11) + j;
			foo.SetElement(i, j, c);
		}
	}
	fprintf(stderr, "calling writeRows\n");
	// populate the TwoDFileObject
	rv = tst.writeRows(&foo, 0, 1, 9);
	fprintf(stderr, "back from writeRows\n");
	// check return value
	if(rv < 0)
	{
		fprintf(stderr, "unable to write to TwoDFileObject\n");
		return 1;
	}
	fprintf(stderr, "calling readRows\n");
	// read back every other row of the file
	bar = tst.readRows(0, 2, 5);
	fprintf(stderr, "back from readRows\n");
	// check return value
	if(bar == NULL)
	{
		fprintf(stderr, "unable to read from TwoDFileObject\n");
		return 1;
	}
	// print out the results of the read
	for(i = 0; i < 5; i++)
	{
		for(j = 0; j < 11; j++)
		{
			c = bar->GetElement(i, j);
			fprintf(stderr, "%d, ", c);
		}
		fprintf(stderr, "\n");
	}

	return 0;
}

Example D shows a Fortran application that creates a 11x9 two-dimensional file consisting of six subfiles. The biggest and smallest subfile do not differ by more than one row and one column, so the biggest subfile is 4x5 and the smallest is 3x4. After creating the file, the program fills it in with a writeBlock call. Finally, the program reads back a subsection of the file with the readElements call.

Example D Fortran 2D file program

	program main
	implicit none

	integer c, i, j, rv, fd, ierr
	integer foo(9,11), bar(9,11)

c	initialize the legion libraries
	call liof_init(ierr)

	write (6,*) 'calling create'
c	create the TwoDFileObject and all of its subfiles
	call liof_2d_create_integers(fd,
	+  'TwoDFileTestFile', 11, 9, 3, 2, ' ', ierr)
	write (6,*) 'back from create'

c	check the return value
	if(ierr.lt.0) then
		write (6,*) 'unable to create TwoDFileObject'
		stop
	endif
c	initialize the array
	do 100 j=1,11
		do 200 i=1,9
			foo(i, j) = (i - 1) * 11 + (j - 1)
200		continue
100	continue
	write (6,*) 'calling writeRows'
c	populate the TwoDFileObject
	call liof_2d_writeElements_integers(fd, foo, 1, 1, 1, 1, 9, 11,
	+                                    11, 9, 1, 1, ierr)
	write (6,*) 'back from writeRows'
c	check the return value
	if(ierr.lt.0) then
		write (6,*) 'unable to write to TwoDFileObject'
		stop
	endif
	write (6,*) 'calling readRows'
c	read back a sub section of the file
	call liof_2d_readElements_integers(fd, bar, 1, 1, 2, 2, 5, 6,
	+                                   11, 9, 1, 1, ierr)
	write (6,*) 'back from readRows'
c	check the return value
	if(ierr.lt.0) then
		write (6,*) 'unable to read from TwoDFileObject'
		stop
	endif
c	print out the results
	do 300 j = 1,6
		do 400 i=1,5
			write (6,*) bar(i,j)
400		continue
300	continue
	stop
	end

6.2.6.2 Synchronous interface

C++:
int TwoDFileObject<TYPE>::create(char *name,
unsigned long rowSize,
unsigned long colSize,
unsigned long numSubRows,
unsigned long numSubCols,
char *vaults_context)

C:
int lio_2d_create_<TYPE>(int *fd, char *name,
unsigned long rowSize,
unsigned long colSize,
unsigned long numSubRows,
unsigned long numSubCols,
char *vaultsContext)

FORTRAN:
subroutine liof_2d_create_<TYPE>(fd, name, rowSize, colSize,
numSubRows, numSubCols, vaultsContext, ierr)
integer fd
character name(*)
integer rowSize, colSize, numSubRows, numSubCols
character vaultsContext
integer ierr

C++:
int TwoDFileObject<TYPE>::open(char *name)

C:
int lio_2d_open_<TYPE>(int *fd, char *name)

FORTRAN:
subroutine liof_2d_open_<TYPE>(fd, name, ierr)
integer fd
character name(*)
integer ierr

C++:
int TwoDFileObject<TYPE>::destroy()

C:
int lio_2d_destroy_<TYPE>(int fd)

FORTRAN:
subroutine liof_2d_destroy_<TYPE>(fd, ierr)
integer fd, ierr

C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readRows(
unsigned long startRow,
unsigned long stride,
unsigned long num)

C:
int lio_2d_readRows_<TYPE>(int fd, <type> *data,
unsigned long startRow,
num, int data_width,
int data_height, int data_x, int data_y)

FORTRAN:
subroutine liof_2d_readrows_<TYPE>(fd, data, startRow,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, stride, num, data_width, data_height
integer data_x, data_y, ierr

C++:
int TwoDFileObject<TYPE>::writeRows(
LegionArray<TYPE> *data,
unsigned long startRow,
unsigned long stride,
unsigned long num)

C:
int lio_2d_writeRows_<TYPE>(int fd, <type> *data,
unsigned long startRow,
unsigned long stride,
unsigned long num,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_writerows_<TYPE>(fd, data, startRow,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, stride, num, data_width, data_height
integer data_x, data_y, ierr

C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readCols(
unsigned long startCol,
unsigned long stride,
unsigned long num)

C:
int lio_2d_readCols_<TYPE>(int fd, <type> *data,
unsigned long startCol,
unsigned long stride,
unsigned long num,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_readcols_<TYPE>(fd, data, startCol,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startCol, stride, num, data_width, data_height
integer data_x, data_y, ierr

C++:
int TwoDFileObject<TYPE>::writeCols(
LegionArray<TYPE> *data,
unsigned long startCol,
unsigned long stride,
unsigned long num)

C:
int lio_2d_writeCols_<TYPE>(int fd, <type> *data,
unsigned long startCol,
unsigned long stride,
unsigned long num,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_writecols_<TYPE>(fd, data, startCol,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startCol, stride, num, data_width, data_height
integer data_x, data_y, ierr

C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readBlock(
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long numCols)

C:
int lio_2d_readBlock_<TYPE>(int fd, <type> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long numCols,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_readblock_<TYPE>(fd, data,
startRow, startCol, numRows, numCols,
data_width, data_height, data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, numRows, numCols
integer data_width, data_height, data_x, data_y, ierr

C++:
int TwoDFileObject<TYPE>::writeBlock(
LegionArray<TYPE> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long numCols)

C:
int lio_2d_writeBlock_<TYPE>(int fd, <type> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long numCols,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_writeblock_<TYPE>(fd, data,
startRow, startCol,
numRows, numCols,
data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, numRows, numCols
integer data_width, data_height, data_x, data_y, ierr

C++:

LegionArray<TYPE> *TwoDFileObject<TYPE>::readElements(
unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long numCol)

C:
int lio_2d_readElements_<TYPE>(int fd, <type> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long numCol,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_readelements_<TYPE>(fd, data,
startRow, startCol,
strideRow, strideCol,
numRow, numCol,
data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, strideRow, strideCol
integer numRow, numCol, data_width, data_height, data_x
integer data_y, ierr

C++:
int TwoDFileObject<TYPE>::writeElements(
LegionArray<TYPE> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long numCol)

C:
int lio_2d_writeElements_<TYPE>(int fd, <type> *data, unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long numCol,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_writeelements_<TYPE>(fd, data,
startRow, startCol,
strideRow, strideCol,
numRow, numCol,
data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, strideRow, strideCol
integer numRow, numCol, data_width, data_height, data_x
integer data_y, ierr

C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readSequential(
unsigned long startElement,
unsigned long num)

C:
int lio_2d_readSequential_<TYPE>(int fd,
<type> *data unsigned long startElement,
unsigned long num,
int data_leng, int data_off)

FORTRAN:
subroutine liof_2d_readsequential_<TYPE>(fd,
data startElement, num,
data_leng, data_off, ierr)
integer fd
<TYPE> data(*)
integer startElement, num, data_leng, data_off, ierr

C++:
int TwoDFileObject<TYPE>::writeSequential(
LegionArray<TYPE> *data,
unsigned long startElement,
unsigned long num)

C:
int lio_2d_writeSequential_<TYPE>(int fd, <type> *data,
unsigned long startElement,
unsigned long num,
int data_leng, int data_off)

FORTRAN:
subroutine liof_2d_writesequential_<TYPE>(fd,
data, startElement,
num, data_leng, data_off, ierr)
integer fd
<TYPE> data(*)
integer startElement, num, data_leng, data_off, ierr

C++:
unsigned long TwoDFileObject<TYPE>::numRows()

C:
int lio_2d_numRows(int fd)

FORTRAN:
subroutine liof_2d_numrows(fd, numRow, ierr)
integer fd, numRow, ierr

C++:
unsigned long TwoDFileObject<TYPE>::numCols()

C:
int lio_2d_numCols(int fd)

FORTRAN:
subroutine liof_2d_numcols(fd, numRow, ierr)
integer fd, numRow, ierr

6.2.6.3 Asynchronous interface

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadRows(
unsigned long startRow,
unsigned long stride,
unsigned long num)

C:
void *lio_2d_nbReadRows_<TYPE>(int fd,
unsigned long startRow,
unsigned long stride, unsigned long num)

FORTRAN:
subroutine liof_2d_nb_readrows_<TYPE>(fd, startRow,
stride, num, req, ierr)
integer fd, startRow, stride, num, req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteRows(
LegionArray<TYPE> *data,
unsigned long startRow,
unsigned long stride,
unsigned long num)

C:
void *lio_2d_nbWriteRows_<TYPE>(int fd, <TYPE> *data,
unsigned long startRow,
unsigned long stride,
unsigned long num,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_nb_writerows_<TYPE>(fd, data,
startRow, stride, num,
data_width, data_height,
data_x, data_y, req, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, stride, num, data_width, data_height
integer data_x, data_y, req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadCols(
unsigned long startCol,
unsigned long stride,
unsigned long num)

C:
void *lio_2d_nbReadCols_<TYPE>(int fd,
unsigned long startCol,
unsigned long stride,
unsigned long num)

FORTRAN:
subroutine liof_2d_nb_readcols_<TYPE>(fd, startCol,
stride, num, req, ierr)
integer fd, startCol, stride, num, req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteCols(
LegionArray<TYPE> *data,
unsigned long startCol
unsigned long stride,
unsigned long num)

C:
void *lio_2d_nbWriteCols_<TYPE>(int fd, <TYPE> *data,
unsigned long startCol,
unsigned long stride,
unsigned long num,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_nb_writecols_<TYPE>(fd, data,
startCol, stride, num,
data_width, data_height,
data_x, data_y, req, ierr)
integer fd
<TYPE> data(*,*)
integer startCol, stride, num, data_width, data_height
integer data_x, data_y, req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadBlock(
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long numCols)

C:
void *lio_2d_nbReadBlock_<TYPE>(int fd,
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long)

FORTRAN:
subroutine liof_2d_nb_readblock_<TYPE>(fd,
startRow, startCol,
numRows, numCols,
req, ierr)
integer fd, startRow, startCol, numRows, numCols, integer req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteBlock(
LegionArray<TYPE> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long numCols)

C:
void *lio_2d_nbWriteBlock_<TYPE>(int fd, <TYPE> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long numRows,
unsigned long numCols,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_nb_writeblock_<TYPE>(fd, data,
startRow, startCol,
numRows, numCols,
data_width, data_height,
data_x, data_y, req, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, numRows, numCols
integer data_width, data_height, data_x, data_y, integer req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadElements(
unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long numCol)

C:
void *lio_2d_nbReadElements_<TYPE>(int fd,
unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long)

FORTRAN:
subroutine liof_2d_nb_readelements_<TYPE>(fd,
startRow, startCol,
strideRow, strideCol,
numRow, numCol, req, ierr)
integer fd, startRow, startCol, strideRow, strideCol
integer numRow, numCol, req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteElements(
LegionArray<TYPE> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long numCol)

C:
void *lio_2d_nbWriteElements_<TYPE>(int fd, <TYPE> *data,
unsigned long startRow,
unsigned long startCol,
unsigned long strideRow,
unsigned long strideCol,
unsigned long numRow,
unsigned long numCol,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_nbWriteelements_<TYPE>(fd, data,
startRow, startCol,
strideRow, strideCol,
numRow, numCol,
data_width, data_height,
data_x, data_y, req, ierr)
integer fd <TYPE> data(*,*)
integer startRow, startCol, strideRow, strideCol
integer numRow, numCol, data_width, data_height
integer data_x, data_y, req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadSequential(
unsigned long startElement,
unsigned long num)

C:
void *lio_2d_nbReadSequential_<TYPE>(int fd,
unsigned long startElement,
unsigned long num)

FORTRAN:
subroutine liof_2d_nb_readsequential_<TYPE>(fd, startElement,
num, req, ierr)
integer fd, startElement, num, req, ierr

C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteSequential(
LegionArray<TYPE> *data,
unsigned long startElement,
unsigned long num)

C:
void *lio_2d_nbWriteSequential_<TYPE>(int fd, <TYPE> *data,
unsigned long startElement,
unsigned long num,
int data_leng, int data_off)

FORTRAN:
subroutine liof_2d_nb_writesequential_<TYPE>(fd, data,
startElement, num, data_leng, data_off, req, ierr)
integer fd
<TYPE> data(*)
integer startElement, num, data_leng, data_off, integer req, ierr

C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::getRead(
TwoDFileRequest *req)

C:
int lio_2d_getRead_<TYPE>(int fd, void *req, <TYPE> *data,
int data_width, int data_height,
int data_x, int data_y)

FORTRAN:
subroutine liof_2d_getread_<TYPE>(int *fd, int *req, int *data,
int *data_width, int *data_height,
int *data_x, int *data_y, int *ierr)
integer fd, req,
<TYPE> data(*,*)
integer data_width, data_height, data_x, data_y, ierr

C++:
int TwoDFileObject<TYPE>::getWrite(TwoDFileRequest *req)

C:
int lio_2d_getWrite_<TYPE>(int fd, void *req)

FORTRAN:
subroutine liof_2d_getWrite_<TYPE>(fd, req, ierr)
integer fd, req, ierr

C:
int lio_2d_free_request(void *req)

FORTRAN:
subroutine liof_2d_free_request(req, ierr)
integer req, ierr

6.3 Terminal I/O

When objects run in the Legion system, they execute on remote hosts. Thus (in the default case), their output (e.g. printfs, writes to stderr, fortran writes to unit 6, etc.) is not visible in the terminal where the main program was started. This situation can make debugging, porting existing code, and many other problems extremely difficult. To address this, the Legion file library supports mechanisms to allow remote objects to use terminal output. The interface to these mechanisms is presented in this section. Note that this interface is run automatically in PVM and MPI, and that there is a similar, smaller, set of routines that can be run in Fortran (below). The basic operation of these mechanisms is based on the use of tty objects, which are objects to which output from remote programs can be directed and from which this output can be obtained and displayed on the screen.

6.3.1 Library interface

The library interface supports two basic styles of performing terminal I/O, each with different features, costs, and benefits.

Both of these styles may be combined arbitrarily. In order to operate the library interface, run-time support in the form of tty objects must be created (explained in "About Legion tty objects" in the Basic User Manual).

The interface for explicit terminal I/O consists of the following functions.

int
legion_tty_init
();

int
legion_tty_write(char *buf, int n);

    Writes n bytes contained in the buffer buf to the current tty object set in the callers environment. Returns the number of bytes written, or -1 on error.

int
legion_puts(char *s);

    Writes the null terminates string starting at s to the current tty object set in the callers environment. Returns the number of bytes written, or -1 on error.

int
legion_printf(char *format, ...);

    Performs formatted I/O in the style of the standard C library function printf. Output is directed to the current tty object set in the callers environment. See the man page for printf for an explanation of the format string.

The interface for the implicit terminal I/O mechanism consists of the following functions.

int
legion_map_stdio();

    This function can be called by a Legion object to map native standard output and standard error (i.e. file descriptors 1 and 2) to the current tty object set in the callers environment. All future writes to stdout and stderr will be mapped to the tty object, until legion_unmap_stdio is called. Note that if the current standard output file descriptor is attached to a terminal this call has no effect. It can thus safely be called by objects that will sometimes operate from the command line and sometimes act as remote objects.

    Returns 0 on success, -1 if no tty object is available in the caller's environment.

int
legion_unmap_stdio();

    Disconnects standard output from the current tty object. Note that legion_unmap_stdio must be called between successive calls to legion_map_stdio in order to map standard output to different tty objects.

    Returns -1 if standard output is not currently connected to a tty object, 0 on success.

int
legion_register_tty_callback(void (*callback)(char *, long));

    This library routine can be called to register a function that will be used to observe output written to the tty object that is currently set in the caller's environment. As data is written to the tty object, the callback specified by this function will be invoked. Each invocation will be passed the length of the data as well as a pointer to the new data. By using this function an object can asynchronously observe the data written to the current terminal object. Note that the interface allows only a single registered callback in a single Legion object, although any number of Legion objects can use this function to observe the same tty object (all data is sent to all registered observers).

    Returns 0 on success, -1 if no tty object is set in the current environment.

int
legion_watch_stdio();

    Whereas legion_register_tty_callback allows a caller to process the data written to a tty object arbitrarily, legion_watch_stdio performs the common-case action of mapping the current tty object's output to the caller's standard output. After this function is called, any data written to the current tty object will be copied to the standard output of the caller. For example, this function might be called at the beginning of the master process so that the output from all of its slaves will be mapped to its own standard output.

    Returns 0 on success, -1 if no tty object is set in the current environment.

    WARNING: legion_watch_stdio is completely asynchronous. If objects (e.g., children of the caller) perform output after the caller has exited, the output will not be displayed to the caller's terminal.

int
legion_disconnect_stdio();

    This call unregisters the current callback if one was specified by legion_register_tty_callback, or disconnects the current tty from the callers standard output if it was connected using legion_watch_stdio.

    Returns 0 on success, -1 on failure.

Some of the above routines can be called from Fortran. The interface is listed below:

    subroutine legionf_tty_init(ret)
    integer ret

    subroutine legionf_map_stdio(ret)
    integer ret

    subroutine legionf_unmap_stdio(ret)
    integer ret

    subroutine legionf_watch_stdio(ret)
    integer ret

    subroutine legionf_disconnect_stdio(ret)
    integer ret

In all cases, the return code of the routine (as described for the C interface above) is stored in ret.

6.3.2 Command-line tools

For the Legion terminal output mechanisms to operate correctly, runtime support in the form of tty objects is created (tty objects are discussed in more detail on "About Legion tty objects" in the Basic User Manual). There are two ways to set up this runtime support:

  1. The standard Legion command legion_tty creates, sets, and watches tty objects. For example:
  2. legion_tty my_stdio

    The name my_stdio is a Legion context name, and can be any name selected by the user. Use legion_tty_off to exit.

  3. Or, you can use a separate command for each step. The legion_create_object command can be used to create a tty object.
  4. legion_create_object -c /class/ttyObjectClass my_stdio

    The legion_set_tty command will set a previously created tty object. The syntax is:

    legion_set_tty <context name>

    For example, to set the tty object created in step 1 (my_stdio) as the tty object for Legion programs you will run in the current shell, execute the following:

    legion_set_tty my_stdio

    You can watch the current tty object set in your shell and observe the data written to it using the legion_tty_watch command. This command attaches to the current tty object and displays its output. The command is blocking: use ^C to exit.


Back to Developer Manual Table of Contents

Directory of Legion 1.5 Manuals