AMS Extensions

Version 3.0

Andrew Hanushevsky

Stanford Linear Accelerator Center

8/26/98

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Acknowledgements

Urs Bertschinger, Objectivity Inc.

for co-designing the initial oofs() interface and modifying AMS v4 to use it.

Karl Quackenbush, Objectivity Inc.

for suggested improvements and modifying AMS v5 to use it.

Marcin Nowak, CERN

for invaluable debugging assistance and helpful modifications.

1 Introduction *

1.1 Acronyms *

1.2 Common Definitions *

1.2.1 Constants *

1.2.2 Limits *

1.2.3 Data Types *

1.2.4 Classes *

1.2.4.1 The oofsError Class *

1.3 Client-Side Definitions *

1.3.1 Constants *

1.3.2 Data Types *

1.3.3 Classes *

1.3.3.1 The oofsSecurity Class *

1.3.4 Opaque Information Interface *

1.4 Server-Side Definitions *

1.4.1 Constants *

1.4.2 Data Types *

1.4.3 Classes *

1.4.3.1 The oofsDesc Class *

1.4.3.2 The oofsDir Class *

1.4.3.3 The oofsFile Class *

2 oofs Interface Definition *

2.1 oofsGetFileSystem() *

2.2 oofsDesc Class Methods *

2.2.1 exists() *

2.2.2 GetSecToken() *

2.2.3 openDir() *

2.2.4 openFile() *

2.2.5 remove() *

2.2.6 rename() *

2.3 oofsDir Class Methods *

2.3.1 close() *

2.3.2 getNext() *

2.3.3 open() *

2.4 oofsError Class Methods *

2.4.1 getErrorInfo() *

2.5 oofsFile Class Methods *

2.5.1 close() *

2.5.2 getmode() *

2.5.3 getsize() *

2.5.4 open() *

2.5.5 read() *

2.5.6 sync() *

2.5.7 truncate() *

2.5.8 write() *

3 Generic Authentication Protocol *

3.1 Application Steps in GAP *

3.2 OCSK Steps in GAP *

3.3 AMS Steps in GAP *

3.4 Security Interface Definition *

3.4.1 oofs_Register_Security() *

3.4.1.1 GetCred() *

4 Opaque Information Protocol *

4.1 Application Steps in OIP *

4.2 OCSK Steps in OIP *

4.3 AMS Steps in OIP *

4.3.1 oofs_set_info() *

5 Defer Request Protocol *

5.1 AMS Steps in DRP *

5.2 OCSK Steps in DRP *

6 Request Redirection Protocol *

6.1 AMS Steps in RRP *

6.2 OCSK Steps in RRP *

7 Appendix A *

7.1 oofs_set() *

  1. Introduction

This document describes Advanced Multithreaded Server (AMS) extensions providing:

The relationships between these components are shown in the following figure.

 

 

 

 

 

 

 

 

 

Major changes occur in the AMS. Prior to the changes, the AMS was distributed as a single module. The new AMS is distributed as two linkable components:

These two components are then linked to two site selectable components:

A functional AMS is built by linking the appropriate four components. The default AMS is normally distributed as an executable module linked to use the native Unix File System (UFS) and a null security service.

The client-side components are relatively straightforward. The Objectivity Kernel internally handles DRP and RRP. Client-supplied programs to transmit authentication and opaque information via registered callback functions implicitly use the GAP and OIP components. Hence, the Kernel is distributed as a single linkable component, as before.

 

    1. Acronyms
      2.

      AMS Advanced Multithreaded Server

      AMSC Advanced Multithreaded Server Collective

      APID Autonomous Partition Identifier

      DRP Defer Request Protocol

      FDID Federated Database Identifier

      GAP Generic Authentication Protocol

      OCSK Objectivity Client-Side Kernel

      OOFS Object Oriented File System

      OIP Opaque Information Protocol

      RRP Request Redirect Protocol

      SO Security Object of class oofsSecurity

      ST Security Token returned by the AMS

       

    2. Common Definitions
      1. Constants
        2.

        // Valid return values from oofs() that return an integer

        #define OOFS_OK 0 // Return code -> All is well

        #define OOFS_ERROR -1 // Return code -> Error occurred

         

      2. Limits

        3. // Longest filename returned by interface (including trailing null)

        #define OOFS_MAX_FILE_NAME_LEN (1024+1)

        // Longest error message string returned (including trailing null)

        #define OOFS_MAX_ERROR_LEN (255+1)

        // Maximum number of bytes of opaque information that can be set

        #define OOFS_MAX_INFO_LEN 4096

        // Maximum number of bytes of security information that can be set

        #define OOFS_MAX_SECTOKEN_LEN 4096

         

      4. Data Types
        5.

        typedef char oofsErrorBuf[OOFS_MAX_ERROR_LEN];

        typedef oofsBuffer oofsOpaqueInfo;

        typedef oofsBuffer oofsCredentials;

        struct oofsErrorStruct // Error information structure

        {

        ooUInt32 code;

        oofsErrorBuf message;

        oofsErrorStruct() {code = 0; message[0] = ‘\0’;}

        };

        struct oofsBuffer // Generic buffer definition

        {

        ooUInt32 size;

        char *data;

        oofsBuffer() {size = 0; data = NULL;}

        };

      5. Classes
        1. The oofsError Class

      class oofsError

      {protected:

      oofsErrorStruct ErrorInfo;

      public:

      int getErrorInfo() {return ErrorInfo.code;}

      int getErrorInfo(oofsErrorStruct &ErrorParm)

      {strcpy(ErrorParm.message, ErrorInfo.message);

      ErrorParm.code = ErrorInfo.code;

      Return ErrorInfo.code;

      }

      }

    3. Client-Side Definitions
      1. Constants
        2.

        // Parameter to the oofs_set_info() function to describe handling

        #define OOFS_SOI_ALWAYS 0 // keep resending information

        #define OOFS_SOI_ONCE 1 // send information only once

        // Operation class codes when calling oofs_GetCred()

        #define OOFS_DIRS 0x0100 // Directory operations

        #define OOFS_META 0x0200 // Metadata operations

        #define OOFS_MISC 0x0400 // Miscellaneous

        #define OOFS_OPCL 0x0800 // Open/close operations

        #define OOFS_RWTS 0x1000 // Read/write operations

        // Operation codes when calling oofs_GetCred()

        #define OOFS_CLOSE 0x01||OOFS_OPCL // OPCL: oofs_close()

        #define OOFS_CLOSEDIR 0x02||OOFS_DIRS // Dirs: not used

        #define OOFS_EXISTS 0x03||OOFS_MISC // Misc: oofs_exists()

        #define OOFS_GETSIZE 0x04||OOFS_MISC // Misc: oofs_getsize()

        #define OOFS_GETMODE 0x05||OOFS_MISC // Misc: oofs_getmode()

        #define OOFS_OPEN 0x06||OOFS_OPCL // OPCL: oofs_open()

        #define OOFS_OPENDIR 0x07||OOFS_DIRS // Dirs: oofs_opendir()

        #define OOFS_READ 0x08||OOFS_RWTS // RWTS: oofs_read()

        #define OOFS_READDIR 0x09||OOFS_DIRS // Dirs: not used

        #define OOFS_REMOVE 0x0a||OOFS_META // Meta: oofs_remove()

        #define OOFS_RENAME 0x0b||OOFS_META // Meta: oofs_rename()

        #define OOFS_SYNC 0x0c||OOFS_RWTS // RWTS: oofs_sync()

        #define OOFS_TRUNCATE 0x0d||OOFS_RWTS // RWTS: oofs_truncate()

        #define OOFS_WRITE 0x0e||OOFS_RWTS // RWTS: oofs_write()

         

      2. Data Types

        3. typedef oofsBuffer oofsSecurityToken;

        struct oofsContextStruct // Info passed to Security Object Creator

        {

        struct sockaddr_in ServerHostAddress; // <- getpeername()

        oofsSecurityToken SecToken;

        ooUInt32 FDID;

        ooUInt32 APID;

        }

        struct oofsRequestStruct // Info passed to Security Object GetCred

        {

        ooUInt32 operation;

        ooUInt32 FDID;

        ooUInt32 APID;

        const char * pathname;

        const char * filename;

        }

         

      3. Classes
        1. The oofsSecurity Class

        class oofsSecurity : virtual public oofsError

        {

        public:

        int getVersion()

        {return OOFS_SEC_SYSTEM_DESC_VERSION;}

        oofsCredentials *GetCred(oofsRequestStruct *);

        // Routine to register the oofsSecurity creator

        int oofs_Register_Security(oofsSecurity *);

         

      4. Opaque Information Interface

      int oofs_set_info(const char *info,

      const int infolen,

      const ooInt flags);

    4. Server-Side Definitions
      1. Constants
        2.

        // Parameter to the oofs_Open() function to set access mode

        #define OOFS_S_IRWXU 0000700 // Owner: read, write, execute perm

        #define OOFS_S_IRUSR 0000400 // Owner: read permission

        #define OOFS_S_IWUSR 0000200 // Owner: write permission

        #define OOFS_S_IXUSR 0000100 // Owner: execute/search permission

        #define OOFS_S_IRWXG 0000070 // Group: read, write, execute perm

        #define OOFS_S_IRGRP 0000040 // Group: read permission

        #define OOFS_S_IWGRP 0000020 // Group: write permission

        #define OOFS_S_IXGRP 0000010 // Group: execute/search permission

        #define OOFS_S_IRWXO 0000007 // Other: read, write, execute perm

        #define OOFS_S_IROTH 0000004 // Other: read permission

        #define OOFS_S_IWOTH 0000002 // Other: write permission

        #define OOFS_S_IXOTH 0000001 // Other: execute/search permission

        // Parameter to the oofs_Open() function to set access type

        #define OOFS_O_RDONLY 0 // open read/only

        #define OOFS_O_WRONLY 1 // open write/only

        #define OOFS_O_RDWR 2 // open read/write

        #define OOFS_O_CREAT 0x100 // open creating the file

        // Current version of the oofs() interfaces

        #define OOFS_FILE_SYSTEM_DESC_VERSION 1

        #define OOFS_SEC_SYSTEM_DESC_VERSION 1

         

      2. Data Types

        3. typedef char oofsFileNameBuf[OOFS_MAX_FILE_NAME_LEN];

        typedef ooInt32 oofsFileCreateMode;

        typedef ooInt32 oofsFileOpenMode;

        typedef ooInt32 oofsXferSize;

        struct oofsCredStruct // Credentials passed to oofs()

        {

        struct sockaddr_in ClientHostAddress; // <- getpeername()

        struct oofsCredentials ClientCredentials;

        }

        struct oofsFileOffset // 64-bit file offset

        {

        ooUInt32 high_offset;

        ooUInt32 low_offset;

        };

         

      3. Classes
        1. The oofsDesc Class
          2.

          class oofsDesc

          {

          public:

          int getVersion() {return OOFS_FILE_SYSTEM_DESC_VERSION;}

          /********** File Functions **********/

          virtual oofsFile *openFile(const char *FileName,

          oofsFileOpenMode,

          oofsFileCreateMode,

          oofsErrorStruct &,

          oofsCredStruct *,

          oofsOpaqueInfo *);

          /********** Directory Functions **********/

          virtual oofsDir *openDir(const char *DirectoryPath,

          oofsErrorStruct &,

          oofsCredStruct *);

          /********** Miscellaneous Functions **********/

          virtual int exists(const char *FileName,

          int &exist_flag,

          oofsErrorStruct &);

          virtual int remove(const char *FileName

          oofsErrorStruct &);

          virtual int rename(const char *FileName1,

          const char *FileName2

          oofsErrorStruct &);

          virtual oofsSecurityToken *GetSecToken(oofsErrorStruct &);

          /********** Miscellaneous Functions + Security **********/

          virtual int exists(const char *fileName,

          int &exists_flag,

          oofsErrorStruct &,

          oofsCredStruct *)

          {return exists(fileName, exists_flag);}

          virtual int remove(const char *fileName,

          oofsErrorStruct &,

          oofsCredStruct *)

          {return remove(fileName);}

          virtual int rename(const char *fileName1,

          const char *fileName2,

          oofsErrorStruct &,

          oofsCredStruct *)

          {return rename(fileName1, fileName2);}

          virtual oofsSecurityToken *GetSecToken

          (oofsErrorStruct &,

          oofsCredStruct *) {return GetSecToken();}

          // destruct called at AMS exit

          void (*destruct)(); // can set to 0, to not destruct

          };

          // Init & return -> FSDesc

          oofsFileSystemDesc *oofsGetFileSystem();

           

        2. The oofsDir Class

          3. class oofsDir : virtual public oofsError

          {protected:

          void *data;

          public:

          oofsDir() {data = 0;}

          /********** Basic Directory Functions **********/

          virtual int open( const char *dir );

          // no more avail when return of 0 and error.code == 0

          virtual const char *getNext();

          virtual int close();

           

          /********** Directory Functions + Security **********/

          virtual int open(const char *dir,

          oofsCredStruct *credentials)

          {return open(dir);}

          virtual const char *getNext(oofsCredStruct *credentials)

          {return getNext();}

          virtual int close(oofsCredStruct *credentials)

          {return close();}

          }; // class oofsDir

        3. The oofsFile Class

class oofsFile : virtual public oofsError

{protected:

void *data;

public:

oofsFile(){data = 0;}

/********** Basic File Functions **********/

virtual int open(const char *fileName,

oofsFileOpenMode omode,

oofsFileCreateMode cmode);

virtual int close();

virtual oofsXferSize read(oofsFileOffset,

char *buffer,

oofsXferSize buffer_size);

virtual oofsXferSize write(oofsFileOffset,

char *buffer,

oofsXferSize buffer_size);

virtual int sync();

virtual int truncate(oofsFileOffset);

virtual int getSize(oofsFileOffset &);

virtual int getMode(oofsFileCreateMode &);

/******* Basic File Functions With Security *******/

virtual int open(const char *fileName,

oofsFileOpenMode openMode,

oofsFileCreateMode createMode,

oofsCredStruct *credentials)

{return open(fileName, openMode, createMode);}

virtual int close(oofsCredentials *credentials)

{return close();}

virtual oofsXferSize read(oofsFileOffset fileOffset,

char *buffer,

oofsXferSize buffer_size,

oofsCredStruct *credentials)

{return read(fileOffset, buffer, buffer_size);

virtual oofsXferSize write(oofsFileOffset fileOffset,

char *buffer,

oofsXferSize buffer_size,

oofsCredStruct *credentials)

{return write(fileOffset, buffer, buffer_size);

virtual int sync (oofsCredStruct *credentials)

{return sync();}

virtual int truncate(oofsFileOffset fileOffset,

oofsCredStruct *credentials)

{return truncate(fileOffset);}

virtual int getSize(oofsFileOffset &fileOffset,

oofsCredStruct *credentials)

{return getSize(fileOffset);}

virtual int getMode(oofsFileCreateMode &CreateMode,

oofsCredStruct *credentials)

{return getMode(CreateMode);}

/* File Functions With Security & Opaque Information */

virtual int open(const char *fileName,

oofsFileOpenMode openMode,

oofsFileCreateMode createMode,

oofsCredStruct *credentials,

oofsOpaqueInfo *opaqueInfo)

{return open(fileName, openMode, createMode, credentials);}

}; // class oofsFile

 

 

  1. oofs Interface Definition
    1. oofsGetFileSystem()

 

oofsDesc *oofsGetFileSystem(); // In

 

Function

Provide the filesystem interface definition.

Parameters

None.

Success

A non-null pointer to the oofsDesc object is returned.

Failure

A NULL (i.e., 0) pointer is returned.

Notes

  1. The oofsGetFileSystem() routine is part of the of the filesystem definition and must be supplied by the underlying filesystem interface. It is called by the AMS to instantiate the filesystem interface.
  2. The oofsGetFileSystem() routine is a constructor function and should perform any required initialization.
  3. The filesystem interfaces are defined in subsequent sections.
    1. oofsDesc Class Methods
      1. exists()

 

int exists(const char *path, // In

int &exists_flag // Out

oofsErrorStruct &einfo, // Out

[, oofsCredStruct *cred ]); // In

 

Function

Determine whether a path or file exists.

Parameters

path

is the fully qualified name of the file to be tested for existence.

exists_flag

is the address of the variable to hold the status of the test. When a success indication is returned, exists_flag will have one of the following values:
    1. the file was not found but the path exists, or
    2. the file was found.

einfo

is the error information structure used when an error occurs.

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

Success

Zero (0) is returned with exists_flag set.

Failure

OOFS_ERROR is returned and einfo.code contains the actual error code while einfo.message contains an optional null terminated string describing the nature of the error.
      2. GetSecToken()

 

oofsSecurity *GetSecToken( oofsErrorStruct &einfo, // Out

[, oofsCredStruct *cred ]); // In

 

Function

Return a security token to be used to initialize client-side security.

Parameters

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

einfo

is the error information structure used when an error occurs.

Success

A pointer to the security token is returned if such a token exists. Otherwise, a NULL pointer is returned.

Failure

OOFS_ERROR is returned and einfo.code contains the actual error code while einfo.message contains an optional null terminated string describing the nature of the error.

Notes

    1. The security token cannot be longer than OOFS_MAX_SECTOKEN_LEN.
    2. The returned pointer points to a static area that may be changed on a subsequent call to GetSecToken(). In a multi-threaded environment, the call to this routine should be protected with a semaphore until the data has been copied to local storage.

 

      2. openDir()

 

oofsDir *openDir( const char *dirname, // In

oofsErrorStruct &einfo, // Out

[, oofsCredStruct *cred]); // In

 

Function

Open a oofsDir object directory for reading.

Parameters

dirname

is the name of the directory that is to be opened. Typically, a fully qualified path is given.

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

Success

A non-NULL pointer to an oofsDir object is returned.

Failure

A NULL pointer is returned and einfo.code contains the actual error code while einfo.message contains an optional null terminated string describing the nature of the error.

Notes

    1. See oofsDir::open() for complete details on the parameters.
      1. openFile()

 

oofsFile *openFile(const char *filename, // In

oofsFileOpenMode open_mode, // In

oofsFileCreateMode create_mode // In

oofsErrorStruct &einfo, // Out

[, oofsCredStruct *cred // In

[, oofsInfoStruct *info ] ]); // In

 

Function

Open an oofsFile object file; optionally creating a corresponding disk file.

Parameters

filename

is the name of the file that is to be opened

open_mode

indicates how the file is to be opened or created (i.e., read, write, or update).

create_mode

holds the access mode bits to be assigned to the file when open_mode indicates that the file is to be created.

einfo

is the error information structure used when an error occurs.

cred

are optional credentials authenticating the remote caller

info

is an optional pointer to a structure describing opaque information that may be of use during the operation.

Success

A non-NULL pointer to an oofsFile object is returned.

Failure

A NULL pointer is returned and einfo.code contains the actual error code while einfo.message contains an optional null terminated string describing the nature of the error.

Notes

    1. See oofsFile::open() for complete details on the parameters.
      2. remove()
        3.

         

        int remove( const char *filename // In

        oofsErrorStruct &einfo, // Out

        [, oofsCredStruct *cred ]); // In

         

        Function

        Remove a file.

        Parameters

        filename

        is the filename of the file to be removed. Normally, a fully qualified path is specified. Asterisks are not allowed in the filename.

        cred

        are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

        einfo

        is the error information structure used when an error occurs.

        Success

        Zero (0) is returned.

        Failure

        OOFS_ERROR is returned and einfo.code contains the actual error code while einfo.message contains an optional null terminated string describing the nature of the error.

         

      4. rename()

 

int rename( const char *old_filename, // In

const char *new_filename, // In

oofsErrorStruct &einfo, // Out

[, oofsCredStruct *cred ]); // In

 

Function

Rename a file.

Parameters

old_filename

is the existing name of a file. Normally, a fully qualified path is specified.

new_filename

is the name that the file is to have. It must be different from old_filename.

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

einfo

is the error information structure used when an error occurs.

Success

Zero (0) is returned.

Failure

OOFS_ERROR is returned and einfo.code contains the actual error code while einfo.message contains an optional null terminated string describing the nature of the error.
    1. oofsDir Class Methods
      1. close()
        2.

         

        int close([ oofsCredStruct *cred ]); // In

         

        Function

        Closes an open oofsDir object directory.

        Parameters

        cred

        are optional credentials authenticating the remote caller. .A null pointer or zero-length credentials indicate that no credentials were supplied.

        Success

        Zero (0) is returned.

        Failure

        OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

      3. getNext()

 

const char * getNext([ oofsCredStruct *cred ]); // In

 

Function

Return the next entry in an open oofsDir object directory.

Parameters

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

Success

A pointer to a local buffer is returned. The buffer contains the next directory entry’s name.

Failure

Zero (0) is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

Notes

    1. When no more directory entries are left a null pointer is returned, the same as result in case of error condition. However, getErrorInfo() will return a zero (0) error code indicating that no error has actually occurred.
      2. open()

 

int open(const char *dirname // In

[, oofsCredStruct *cred]); // In

 

Function

Open a oofsDir object directory for reading.

Parameters

dirname

is the name of the directory that is to be opened. Typically, a fully qualified path is given.

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

Success

Zero(0) is returned.

Failure

OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.
    2. oofsError Class Methods
      1. getErrorInfo()

 

void getErrorInfo([oofsErrorStruct &einfo]); // Out

 

Function

Return error information about the last oofs() operation..

Parameters

einfo

is an optional pointer to the structure that is to hold the error information.

Success

The error code number is returned if einfo.

Failure

The function cannot fail.

Notes

    1. The getErrorInfo() method may be called at any time and any number of times. If no error has occurred, a code of zero is returned. Otherwise, the actual code from the last operation is returned.
    2. You must supply a structure argument if you wish to obtain the error message associated with the error code.
    1. oofsFile Class Methods
      1. close()
        2.

         

        int close([ oofsCredStruct *cred ]); // In

         

        Function

        Close an open oofsFile object’s file.

        Parameters

        cred

        are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

        Success

        Zero (0) is returned.

        Failure

        OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

         

      3. getmode()
        4.

         

        int getmode(oofsFileCreateMode &mode // Out

        [, oofsCredStruct *cred ]); // In

         

        Function

        Return an oofsFile object’s file creation mode.

        Parameters

        mode

        is a pointer to a variable that is to hold the file’s creation mode (i.e., read-write-execute permission bits). The creation mode is returned in POSIX format and only upon success. The returned value may not be meaningful if the file is protected by an access control list.

        cred

        are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

        Success

        Zero (0) is returned with mode set.

        Failure

        OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

      5. getsize()
        6.

         

        int getsize(oofsFileOffset &flen // Out

        [, oofsCredStruct *cred ]); // In

         

        Function

        Return an oofsFile object file’s current size.

        Parameters

        flen

        is a pointer to a variable to hold the file’s current size. The size is returned only upon success.

        cred

        are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

        Success

        Zero (0) is returned with flen set.

        Failure

        OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

      6. open()

 

int open(const char *filename, // In

oofsFileOpenMode open_mode, // In

oofsFileCreateMode create_mode // In

[, oofsCredStruct *cred // In

[, oofsInfoStruct *info ] ]); // In

 

Function

Open an oofsFile object file; optionally creating a corresponding disk file.

Parameters

filename

is the name of the file that is to be opened. Typically, a fully qualified path is given.

open_mode

indicates how the file is to be opened (i.e., read, write, or update). If open_mode indicates that the file is to be created, the file is also opened in update mode. The following are valid mode values:

OOFS_O_RDONLY - open file for reading

OOFS_O_WRONLY - open the file for writing

OOFS_O_RDWR - open the file for reading and writing (i.e., update)

OOFS_O_CREAT - create the file and open with OOFS_O_RDWR

create_mode

holds the access mode bits to be assigned to the file when open_mode indicates that the file is to be created. The mode must be specified in POSIX format (.e.g., 744 corresponds to rwx--r—mode).

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

info

is an optional pointer to a structure describing opaque information that may be of use during the operation. Opaque information is only meaningful to oofs() routines and is passed with modification from the client. A null pointer or a zero length opaque information indicates that opaque information was not supplied.

 

Success

Zero (0) is returned.

Failure

OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

Notes

    1. If the file already exists and OOFS_O_CREAT is specified, an error is returned.
      2. read()
        3.

         

        oofsXferSize read(oofsFileOffset offset, // In

        char *buffer, // Out

        oofs XferSize buffer_size // In

        [, oofsCredStruct *cred ]); // In

         

        Function

        Read zero or more bytes from an open oofsFile object’s file.

        Parameters

        offset

        is the absolute offset, origin 0, into the file where the read is to begin.

        buffer

        is a pointer to a buffer that is to hold the contents of the file after the read completes

        buffer_size

        is the size of the actual buffer. No more than the specified number of bytes are actually read from the file.

        cred

        are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

        Success

        The actual number of bytes that were read.

        Failure

        OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

      4. sync()
        5.

         

        int sync([ oofsCredStruct *cred ]); // In

         

        Function

        Verify that all data is committed to permanent media.

        Parameters

        cred

        are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

        Success

        Zero (0) is returned.

        Failure

        OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

      6. truncate()
        7.

         

        int truncate(oofsFileOffset file_size // In

        [, oofsCredStruct *cred ]); // In

         

        Function

        Set the size of a file.

        Parameters

        file_size

        is the new size of the file. If the new size is smaller than the current size, the file is reduced in size and the excess bytes are discarded. If the new size if greater than the current size, the file is extended to the new size with binary zeroes.

        cred

        are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

        Success

        Zero (0) is returned.

        Failure

        OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

      8. write()

 

oofsXferSize write(oofsFileOffset offset, // In

const char *buffer, // In

oofs XferSize buffer_size // In

[, oofsCredStruct *cred ]); // In

 

Function

Write zero or more bytes into an open oofsFile object’s file.

Parameters

offset

is the absolute offset, origin 0, into the file where the write is to begin.

buffer

is a pointer to a buffer that is to hold the data to be written into the file.

buffer_size

is the size of the actual buffer. No more than the specified number of bytes are actually written to the file.

cred

are optional credentials authenticating the remote caller. A null pointer or zero-length credentials indicate that no credentials were supplied.

Success

The actual number of bytes that were written are returned.

Failure

OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

Notes

  1. Indication of errors may be delayed until some future operation on the file.
  1. Generic Authentication Protocol

Generic Authentication Protocol (GAP) allows security information (e.g., credentials) to be transparently transferred from an Objectivity AMS client to the AMS server using a generic mechanism that can be easily modified to use site-specific security. GAP is supported only for AMS mediated connections and is suitable for private-key mechanisms such as Kerberos and Windows NT, and public-key mechanisms such as PGP. It consists of three site-definable client-side functions and a new client-side interface.

The following general steps comprise GAP:

The following diagram shows the basic sequence, numbered as above.

 

 

 

    2. Application Steps in GAP
  1. The application program must instantiate it’s definition of the oofsSecurity class.
  2. The application program must then call oofs_Register_Security() prior to opening any federated databases. An example follows.

    3.  

    Class mySecurity : public oofsSecurity

    {

    // Actual security implementation

    }

    oofsSecurity *NewSecurityObject(oofsContextStruct *info)

    {

    // Perform any required actions

    return new mySecurity(info);

    }

    if (oofs_RegisterSecurity(NewSecurityObject))

    Fatal_Error();

     

  3. Once the routines have been registered, the OCSK will call the routines whenever needed in support of GAP.

 

    2. OCSK Steps in GAP

There is one Security Object (SO) for each distinct AMS. An SO is created by the routine passed as the argument to oofs_RegisterSecurity(). Therefore, for each new AMS that is contacted, the OSCK follows these steps:

  1. The OSCK obtains a Security Token (ST) from the target AMS by requesting that the AMS return the results of

    2. oofsDesc::oofsGetSecToken().

  2. The ST, the AMS IP-address, FDID, and APID are assembled in the oofsContextStruct structure and passed to the routine registered via oofs_RegisterSecurity().The called routine must create a new SO by instantiating the appropriate class and returning the instantiated object of that class. The class constructors and destructors are responsible for performing initialization and cleanup.

For each request sent to an existing AMS, the OSCK follows these steps using the methods provided by the SO associated with the target AMS:

  1. The operation code, FDID, APID, pathname, and filename pointers are appropriately set in the oofsRequestStruct (see GetCred() for details).
  2. SO.GetCred() for the target AMS is invoked with oofsRequestStruct as an argument. The invoked method must return failure or the appropriate credentials, which may be null.
  3. The OSCK passes the returned credentials, if any, to the AMS along with the database request.
  4. The previous steps continue until there is no need to communicate with the target AMS. At this point, the associated SO is destroyed via delete when it is no longer needed.

 

    2. AMS Steps in GAP
  1. Upon receipt of a request, the AMS determines if a credentials structure was passed.
  2. If none was passed. AMS either initializes the oofsCredStruct with ClientCredentials.size and ClientCredentials.data both set to zero or passes a null pointer instead of a pointer to an oofsCredStruct structure.
  3. If credentials were passed, AMS recreates the credentials in oofsCredStruct as they existed at the OSCK after the GetCred() call.
  4. In either case, if an oofsCredStruct is passed, the ClientHostAddress portion is appropriately filled out.
  5. A pointer oofsCredStruct, which may be null, is passed to the appropriate oofs() routine.
  6. After the AMS completes all oofs() calls, it can free oofsCredStruct in the appropriate way.
    1. Security Interface Definition
      1. oofs_Register_Security()

 

int oofs_Register_Security(oofsSecurity *CreateSecObj); // In

 

Function

Register security callback functions.

Parameters

CreateSecObj

is the routine to be called to create a new security object.

Success

Zero is returned.

Failure

A negative one (-1) is returned.

Notes

  1. The oofs_Register_Security() routine is part of the OCSK. It must be called by the application, prior to opening a federated database, in order to register the security interface.
  2. The supplied routine is called for each AMS that with which the OCSK will establish communications. While typically there is logically a 1-to-1 correspondence between each call and each AMS, CreateSecObj may actually be called any number of times.

 

        1. GetCred()

 

oofsCredStruct * GetCred(oofsRequestStruct request) // In

 

Function

Obtain credentials for a data operation.

Parameters

request

describes the nature of the request. Certain fields are meaningful for certain operations. See the notes for details.

Success

A non-zero pointer to a credentials structure is returned. The data in the structure is sent to the AMS along with the request.

Failure

OOFS_ERROR is returned. Use getErrorInfo() to obtain the actual error code and, optionally, a null terminated string describing the nature of the error.

Notes

  1. GetCred() is not invoked unless oofsRegisterSecurity() was called.
  2. If GetCred() is not called, null credentials are sent to the AMS.
  3. After GetCred() is called for an oofs_opendir() operation, the returned credentials may also be used for subsequent oofs_readdir() and oofs_closedir() operations.
  4. The following table details the contents of the oofsRequestStruct based on the contents of the operation field. Grayed rows indicate that the call for the operation may be omitted by the OCSK and credentials obtained from previous related call may be used by the AMS for these operations.

 

operation

fdid

apid

path

fname

OOFS_CLOSE

present

present

null

null

OOFS_CLOSEDIR

present

present

null

null

OOFS_EXISTS

present

present

null

null

OOFS_GETSIZE

present

present

null

null

OOFS_GETMODE

present

present

null

null

OOFS_OPEN

present

present

present

present

OOFS_OPENDIR

present

present

present

null

OOFS_READ

present

present

null

null

OOFS_READDIR

present

present

null

null

OOFS_REMOVE

present

present

present

present

OOFS_RENAME

present

present

present

present

OOFS_SYNC

present

present

null

null

OOFS_TRUNCATE

present

present

null

null

OOFS_WRITE

present

present

null

null
  1. Opaque Information Protocol

    2. Opaque Information Protocol (OIP) allows arbitrary information (e.g., performance hints) to be transparently transferred from an Objectivity AMS client to the AMS server. OIP is supported only for AMS mediated connections. It consists of one new client-side function, oofs_set_info(). Opaque information is sent to the oofs_open() function on the AMS side.

     

    1. Application Steps in OIP
  1. The application program must initialize a character array with information appropriate to the AMS-side filesystem being used.
  2. The application program must then call oofs_set_info(), passing it a pointer to the character array along with it’s effective length prior to opening the database(s) associated with the information. The application has an option of associated a single (OOFS_SOI_ONCE) database open with the information or all subsequent opens (OOFS_SOI_ALWAYS), using the flags (i.e., third) parameter. An example follows.

    3.  

    char info[256];

    ·

    · // Fill in the info array, as needed

    ·

    if (oofs_set_info(info, sizeof(info), OOFS_SOI_ONCE))

    Fatal_Error();

    ·

    · // Open objectivity database, sending it the info.

    ·

     

  3. Information set by oofs_set_info() is used only for the first explicit database open when OOFS_SOI_ONCE is passed. Otherwise, the same information is used for all subsequent database opens.
  5. The information is considered current until replaced by a another call to oofs_set_info() (i.e., the supplied information is copied to an internal buffer). Any number of calls may be made to oofs_set_info() with information not exceeding OOFS_MAX_INFO_LEN bytes.
  6. The application may nullify opaque information processing by simply calling oofs_set_info() with either a null character array pointer or with the length set to zero.

 

    1. OCSK Steps in OIP
  1. Opaque information is only support for open operations, regardless of cause.
  2. Prior to sending an open() request to AMS, OCSK checks to see if any oofs_set_info() information exists (i.e., a non-null buffer pointer with a non-zero length exists).
  3. If opaque information exists, the length and data are marshaled as (int), and (char), respectively, on to the AMS protocol stream in a suitable way.
  4. Opaque information is passed to the AMS on any request that would result in an AMS oofs_open() call, regardless of the reasons for that call.
  5. If OOFS_SOI_ONCE is associated with the opaque information, the information is discarded after it has been sent (i.e., it is only sent once). Otherwise, an internal copy is maintained for subsequent transmission.

 

    1. AMS Steps in OIP
  1. Upon receipt of a request, the AMS determines if oofs_set_info() information was passed.
  2. If any was passed, it creates an oofsBuffer structure with that information and passes it to the requested oofs_open() call(s. Otherwise, a null pointer is passed as the fifth parameter.
    3.
  3. Opaque information is passed to every oofs_open() initiated by the incoming request.
  4. After the AMS completes all oofs_open() calls, it discards the information.
      2. oofs_set_info()

 

int oofs_set_info(const char *info, // In

const int infolen, // In

const ooInt flags);, // In

 

Function

Set opaque information for subsequent transmission to the AMS.

Parameters

info

points to a buffer containing the information to be sent to the AMS, If the pointer is NULL, current opaque information, if any, is discarded and no information is subsequently sent.

infolen

is the length of the information to be sent to the AMS, If the length is zero, current opaque information, if any, is discarded and no information is subsequently sent.

flags

indicates how the opaque information is to be handled:

OOFS_SOI_ALWAYS - always send the information on subsequent open and create requests.

OOFS_SOI_ONCE - send the information on the subsequent open or create request and discard the information afterwards (i.e., only send it once).

Success

OOFS_OK (0) is returned.

Failure

OOFS_ERROR is returned.

Notes

  1. The oofs_set_info() routine is part of the OCSK. It must be called by the application in order to establish the information that is to be sent.
  2. The oofs_set_info() routine may be called at any time. A new invocation simply replaces opaque information established by the previous call, if any.
  3. Opaque information is only supplied to the AMS-side oofs_open() routine.
  1. Defer Request Protocol

    2. Defer Request Protocol (DRP) allows an AMS server to control the timeout of a client’s request. It is included as part of the AMS/OCSK protocol to allow the use of hierarchical filesystems with highly variable latencies (e.g., microseconds to several minutes).

     

    1. AMS Steps in DRP
  1. At the end of any failing oofs() operation, the oofsGetErrorInfo() returns the reason for the failure, represented as an error code and an optional ASCII character string. Failure indication is defined by each oofs() function as follows:

    2.  

    Function

    Failure

    Return

    Function

    Failure

    Return

    oofs_close()

    -1

    oofs_read()

    -1

    oofs_closedir()

    -1

    oofs_readdir()

    -1

    oofs_exists()

    -1

    oofs_remove()

    -1

    oofs_getmode()

    -1

    oofs_rename()

    -1

    oofs_getsize()

    -1

    oofs_sync()

    -1

    oofs_open()

    0

    oofs_truncate()

    -1

    oofs_opendir()

    0

    oofs_write()

    -1
  2. Operations in greyed boxes are not supported for DRP. These functions will never request a deferal.
  3. If the error code returned by getErrorInfo() is zero and the returned text string starts with the string "!wait", then the function is requesting that the client retry the operation after a certain amount of time. The number of seconds to wait follows !wait, separated by a single space. For example:

    4. !wait 300

    indicates that the client should wait 300 seconds then reissue the same request to the same AMS.

  4. The server uses this information to provide the appropriate indication to the client. The client must retry the request on the same AMS after waiting the specified number of seconds.

 

    2. OCSK Steps in DRP
  1. The OCSK must be prepared to receive a retry request following any interaction with the AMS. Therefore, all data sent to the AMS must be kept until an actual indication of success or failure is received.
  2. Should a retry request be received, the OCSK must wait the specified number of seconds and then send the same request to the same AMS.

3. The OCSK should treat the retry request as an error if any of the following occurs:

a) The number of seconds is less than 1.

b) The number of seconds is greater than 9999.

c) The number of seconds is not a whole number.

4. Retry errors are to be treated as AMS errors and standard error recovery procedures are to be applied.
  1. Request Redirection Protocol

Request Redirection Protocol (RRP) allows cooperating AMS’s to perform dynamic load balancing using a simple redirection scheme. A group of cooperating AMS’s, or an AMS Collective (AMSC), is logically treated as a single AMS by the client. There is one distinguished member of the collective and all database catalogue information is tied to the distinguished member (i.e., server). This minimizes the administrative impact on client operations and database descriptions. Therefore, a client need not know the composition of the collective and, indeed, that composition is free to change at any time without impacting any database operations that the client may perform.

The following figure illustrates an AMS collective and a simple redirection interaction.

 

 

Redirection allows an AMS to direct a client to a more suitable AMS for processing the client’s request. Because the server redirects the request, it is possible to implement a highly flexible and scalable dynamic load-balancing scheme than would otherwise be possible by other mechanisms. For instance, the database load may be balanced using one or more of the following criteria, among others:

RRP also allows the AMS to dynamically replicate databases on demand. For instance, should a databases become highly used, the AMS could replicate the database at one or more other sites and redirect clients to other copies. Once the database becomes less heavily used, the extra copies can be eliminated to save space.

Dynamic load balancing does not alter any current multi-AMS protocols such as the Data Replication Option (DRO) or the Fault Tolerant Option (FTO). This is because the collective is defined outside the scope of such protocols. Furthermore, each AMSC member is effectively interchangeable in the context of the collective. Nevertheless, it is possible to implement collectives in ways that render DRO and FTO unusable.

The collective is responsible for providing update synchronization should any AMSC members allow write operations. Should a client wish to update a database and a modifiable database replica exists outside of the collective, it is the application’s responsibility to explicitly indicate that write operations will occur before performing any transactions against the database.

Redirection of requests is only supported for operations against a closed database (i.e., operations that use a filename instead of a filehandle). This includes the open() and opendir() requests.

 

    2. AMS Steps in RRP
  1. At the end of any failing oofs() operation that takes a filename as an argument, the oofsErrorStruct parameter contains the reason for the failure, represented as an error code and an optional ASCII character string. Failure indication is defined by each oofs() function as follows:
    2.

     

    Function

    Failure

    Return

    Function

    Failure

    Return

    oofs_close()

    -1

    oofs_read()

    -1

    oofs_closedir()

    -1

    oofs_readdir()

    -1

    oofs_exists()

    -1

    oofs_remove()

    -1

    oofs_getmode()

    -1

    oofs_rename()

    -1

    oofs_getsize()

    -1

    oofs_sync()

    -1

    oofs_open()

    0

    oofs_truncate()

    -1

    oofs_opendir()

    0

    oofs_write()

    -1
  2. Operations in greyed boxes are not supported for RRP. These functions will never request a redirection.
    3.
  3. If the oofsErrorStruct.code is zero and the text string in oofsErrorStruct.message starts with the string "!try", then the function is requesting that the client retry the operation at another AMS. The location of the alternate AMS follows !try, separated by a single space, and is either the DNS name or the IP address of the alternate AMS. For example:

    4. !try abh.slac.stanford.edu

    indicates that the client should reissue the same request to the AMS located on host "abh.slac.stanford.edu".

  4. The server may use this information to provide the appropriate indication to the client or may simply pass the error text-string to the client for resolution. The mechanism actually used is immaterial as long as the redirection occurs.

 

    2. OCSK Steps in RRP
  1. The OCSK must be prepared to receive a redirect request following any interaction with the AMS. Therefore, all data sent to the AMS must be kept until an actual indication of success or failure is received.

2. Should a redirect request be received, the OCSK must send the same request to the specified AMS.

3. The OCSK should treat the redirect request as an error if any of the following occurs:

a) The redirect specifies the same AMS.

b) More than 255 consecutive redirect requests have been received.

c) The specified AMS does not exist or refuses a connection.

4. Redirection errors are to be treated as AMS errors and standard error recovery procedures are to be applied.

 

 

 

 

 

 

 

 

 

  1. Appendix A

    2. Local Extensions to the oofs() Interface

    1. oofs_set()

 

int oofs_set( oofsFileDesc fd, // In

const char *options, // In

oofsErrorStruct &einfo) // Out

 

Function

Set oofs() options for a file or for the system..

Parameters

fd

is the file descriptor returned by oofs_open() or oofs_opendir() to which the options are to apply. If fd is null, then the options apply globally and affect all future file descriptors.

options

is a null terminated string of single letter, space-separated, option letters. Valid letters are:

l - log data transfer statistics into SYSLOG when oofs_close() or

oofs_closedir() is called.

p - print data transfer statistics on STDERR when oofs_close() or

oofs_closedir() is called.

t - race execution on STDERR.

Each letter may be optionally preceded by a plus (the default) to turn on the option or a minus, to turn off the option.

einfo

is the error information structure used when an error occurs.

Success

Zero (0) is returned.

Failure

OOFS_ERROR is returned and einfo.code contains the actual error code while einfo.message contains an optional null terminated string describing the nature of the error.

Notes

  1. The oofs_set() interface is not part of the AMS-defined set of interfaces. It is provided to enable externally linked programs additional control over the interface for debugging and performance monitoring purposes.
  2. Default options can be provided on a global level by placing the option string (i.e., the one normally passed via options) in the oofs_DEFAULTS environmental variable.
  3. Setting global options (i.e., when fd is null) is not a thread-safe operation. Global option must be set by a single or interlocking threads.