Appendix B. DMF User Library (libdmfusr.so)
Prev |  | Next

## Appendix B. DMF User Library (`libdmfusr.so`)

The DMF distributed command feature is available with DMF version 2.7 and later. This appendix presents an overview of the feature, a summary of data types, and a summary of user-accessible API subroutines.

## Overview

The distributed command feature allows DMF commands to execute on a host other than the host on which the DMF daemon is running. A host that imports DMF-managed filesystems from the DMF daemon host machine can execute the `dmput`, `dmget`, `dmls`, `dmfind`, `dmattr`, and `dmcopy` commands locally.

As part of the distributed command feature, the DMF user commands listed above were radically re-designed to communicate with a process named `dmusrcmd` instead of directly with the DMF daemon. The DMF user commands are no longer installed as `setuid root` processes. Rather, the `dmusrcmd` process is executed as `setuid root` and performs all of the validity checks and communicates, ultimately, with the DMF daemon.

For the DMF user commands to communicate in an efficient and consistent manner with the `dmusrcmd` process, the DMF user library, `libdmfusr.so`, must be accessed. This is a shared object library (DSO) that is installed in `/usr/{lib|lib32|lib64}` and to which each of the DMF user commands is linked for its protocol-based communications.

As a feature of this re-design, the subroutines that comprise the DMF user command application program interface (API) are now available to user-written programs simply by linking to `libdmfusr.so`. Sites can now design and write their own custom DMF user commands, which eliminates the need to use wrapper scripts around the DMF user commands.

The underlying design of the API calls for the user command to make contact with a `dmusrcmd` process by creating an opaque 'context' object via a call to the API. This context is then used as a parameter on each function (put, get, fullstat, or copy) API call. The context is used by each API routine to perform the requested operation and to correctly return the results of the operation to the command.

In addition to the library, the `libdmfusr.h` and `dmu_err.h` header files are provided, which are required for a site to effectively create their own commands. Both header files are installed in `/usr/include/dmf`. The `libdmfusr.h` file contains all of the object and function prototype definitions required by the API subroutine calls. The `dmu_err.h` file contains all of the API error code definitions. Along with each error code definitions is a text string that is associated with each of the error codes. This text string is the same message that is generated automatically when the error occurs as part of the `DmuErrorInfo_t` object described below (see “DmuErrorInfo_t”). The text string is included in the file as informational only, and is not accessible by a program that includes `dmu_err.h`.

Each type of function request (put, get, fullstat, or copy) can be made via a synchronous or an asynchronous API subroutine call. The synchronous subroutine calls do not return to the caller until the request has completed, either successfully or unsuccessfully. These synchronous subroutines return an error object to the caller that can be processed to determine the success or failure of the call. If an application is making more than one call, these calls are obviously going to perform less efficiently than their asynchronous counterparts because of the serial nature of their activity.

The asynchronous subroutine calls return immediately to the caller. The return code of these asynchronous routines indicate whether the request was successfully forwarded to `dmusrcmd` for processing. A successful return allows the calling program to continue its own processing in parallel with the processing being performed by `dmusrcmd` (or the daemon) to complete the request. If the request was successfully forwarded, a request ID that is unique within the scope of the opaque context is returned to the caller. It is the responsibility of the caller to associate the request id with the correct completion object (described in “DmuCompletion_t”) to determine the eventual result of the original request.

There are several different API subroutine calls for processing asynchronous request completion objects. The user can choose to be simply notified when all requests have completed, without doing any processing of the return status of each request. The user can also choose to process the return status of each request, one at a time, in the order in which they complete, or in the order in which they were sent (request ID order), or the user can, by request ID, synchronously wait on an individual asynchronous request's completion.

The API includes well-defined protocols that it uses to communicate with the `dmusrcmd` process. These protocols make use of the `pthreads(5)` mechanism and as such, any user application program making use of the API via `libdmfusr.so` will also need to link to the shared object library, `libpthread.so`, via the `-lpthread` compiler option (`cc(1)`or `CC(1)`) or loader option (`ld(1)` or `rld(1)`) option.

The API can return different types of objects to the callers of many of the API subroutines by passing the addresses of the objects in subroutine parameters. Many of these objects have been created by allocating new memory for them through the use of the `malloc` command. The API includes several subroutines that will free the memory used by these objects when the caller is through with them, and they are defined below in “Memory Management Subroutines”. It is up to the caller to make use of these subroutines, however, if memory leakage is a concern.

In many cases the API subroutines pass the address of an object back to the caller by setting a '**' pointer accordingly. If errors occur and the subroutine is unable to complete its task, the address returned may be NULL. It is up to the caller to check the validity of an object's address before using it to avoid causing a `SIGSEGV` fault in the application program.

## Data Types

The data types described in this section are defined in `libdmfusr.h`. For the most up-to-date definitions of each of these types, see the `libdmfusr.h` file. The following information is provided as a general description and overall usage outline.

### `DmuAllErrors_t`

This object provides the caller with as much information regarding errors as is practical. The complex nature of the API and its communications allows for many types of errors, and several locations (processes) in which they can occur. For example, a request might fail in the API, in the `dmusrcmd` process, or in the DMF daemon.

This object may contain 0 or more `DmuErrorInfo_t` objects (see “DmuErrorInfo_t”).

### `DmuByteRange_t`

This object defines a range of bytes that are to be associated with a put or get request. The fields and their definitions are as follows:

`offset` |  | Starting offset in bytes of the range in the file.
`size` |  | Size in bytes of the range.

Currently, only offset 0 and size 0 (indicating the whole file) are supported as valid definitions.

### `DmuByteRanges_t`

This object defines a set of `DmuByteRange_t` objects that are to be associated with a put or get request. The fields and their definitions are as follows:

`rounding` |  | Rounding method to be used to validate range addresses. Only `DMU_RND_NONE` is currently defined.
`num_ranges` |  | Number of `DmuByteRange_t` objects in the `ranges` field. Currently, only a single range is allowed.
`ranges` |  | A pointer to an array of `DmuByteRange_t` objects. Currently, only a single element array is allowed.

Example: In the current API, define a `DmuByteRanges_t` as follows:

    DmuByteRanges_t ranges = {DMU_RND_NONE, 0, NULL};

or

    DmuByteRange_t	range = {0, 0};
    DmuByteRanges_t ranges = {DMU_RND_NONE, 1, &range);

### `DmuCompletion_t`

This object is returned by one of the API request completion routines (see “Request Completion Subroutines”) with the results of an asynchronous request.

The `request_id` field can be used to associate the completion object with an asynchronous request that was previously issued. This value coincides with the request ID value that any of the asynchronous routines return to the user.

The `ureq_data` field is request-type specific, and API routines are defined below (see “Fullstat Requests”) to help the application process the object (that is, to extract the `DmuFullstat_t` information from a fullstat completion). This field has no meaning for put, get, or copy requests.

The `reply_code` field has the overall success or failure status of the request. If this value is `DmuNoError` , the request was successful. If not, the `allerrors` field should be checked for the appropriate error information.

The `allerrors` field (type `DmuAllErrors_t` , defined previously) contains the error information for a failed request.

### `DmuCopyRange_t`

This object defines a range of bytes that are to be associated with a copy request. The fields and their definitions are as follows:

`src_offset` |  | Starting offset in bytes of the range in the source file to be copied.
`src_length` |  | Length in bytes of the range to be copied.
`dst_offset` |  | Starting offset in bytes in the destination file to which the copy is sent.

### `DmuCopyRanges_t`

This object defines a set of `DmuCopyRange_t` objects that are to be associated with a put or get request. The fields and their definitions are as follows:

`rounding` |  | Rounding method to be used to validate range addresses. Only `DMU_RND_NONE` is currently defined.
`num_ranges` |  | Number of `DmuCopyRange_t` objects in the `ranges` field. Currently, only a single range is allowed.
`ranges` |  | A pointer to an array of `DmuCopyRange_t` objects. Currently, only a single element array is supported.

### `DmuErrHandler_f`

This type defines a user-specified error handling subroutine. Many of the API subroutines may result in the receipt of error information from the `dmusrcmd` process or the DMF daemon in the processing of the request. As these errors are received, they are formatted into a `DmuErrorInfo_t` object (see “DmuErrorInfo_t”) and are generally returned to the caller either via a calling parameter or as part of a `DmuCompletion_t` object.

In addition, however, if the error occurs in the course of processing internal protocol messages, the `DmuErrorInfo_t` object can also be passed into the `DmuErrHandler_f`, which the caller defined when the opaque context was created.

As part of the `DmuCreateContext()` API subroutine call, the caller can specify a site-defined `DmuErrHandler_f` routine, or the caller can use one of the following API-supplied routines:

`DmuDefErrHandler` |  | Outputs the severity of error and the error message associated with the error to `stderr`.
`DmuNullErrHandler` |  | Does nothing with the error.

### `DmuError_t`

This is the type that most of the API subroutines pass as a return code. The definition `DmuNoError` is the general success return code.

### `DmuErrorInfo_t`

This object contains the information about a single error occurrence. Included are the error code, which might or might not be meaningful to an application, the originator of the error (API, `dmusrcmd`, daemon), a severity code, and perhaps most importantly, an ACSII message that can be displayed.

### `DmuFhandle_t`

This object contains the ASCII representation of the file `fhandle` as it is known on the host on which the file's filesystem is native.

### `DmuFullstat_t`

This rather lengthy object is a user-accessible version of the internal DMF `fullstat` object. It contains all of the basic `stat(2)` information regarding the file, as well as all of the DMAPI related fields.

### `DmuReplyOrder_t`

This type is used to select the order in which asynchronous replies are to be returned by the API reply processing subroutines defined in the following list.

`DmuAnyOrder` |  | Return in the order the replies are received.
`DmuReqOrder` |  | Return in the order the requests were issued.

### `DmuReplyType_t`

This type is used to select the type of reply that an API can receive after sending a request. All requests will receive a final reply when the `dmusrcmd` process has completed processing the request whether it was successful or not.

The valid definitions are:

`DmuIntermed` |  | Intermediate reply. An informational message to alert the caller that the request is being processed and may not complete for some time. An example of this is the intermediate reply that is sent when a put request has been forwarded to an MSP or library server for processing and that the completion reply is deferred until that operation is complete.
`DmuFinal` |  | Final reply for the request.

This definition is used to specify the types of replies that some of the reply processing routines defined below are to consider.

### `DmuReqid_t`

This type is used to describe the request identifier returned to the caller for a successful asynchronous function call.

### `DmuRounding_t`

This is an enum that specifies the kind of address manipulation that the caller would like performed on his DMF put/get/copy file access requests:

`DMU_RND_NONE` |  | Do none.
`DMU_RND_IN` |  | Not yet supported.
`DMU_RND_OUT` |  | Not yet supported.
`DMU_RND_MAX` |  | Not yet supported.

## User-Accessible API Subroutines

This section describes the following types of user-accessible API subroutines:

- Context manipulation

- DMF daemon request

- Request completion

- Memory management

### Context Manipulation Routines

This section describes context manipulation routines.

#### `DmuCreateContext` Subroutine

The `DmuCreateContext` subroutine creates an opaque context for the API to use to correctly communicate with the `dmusrcmd` process. This routine should be the first API subroutine called by a DMF user command. Not only is the context created, but the communication channel to the `dmusrcmd` process is initialized. The code is as follows:

    extern DmuError_t
    DmuCreateContext(
                    void                **dmuctxt,
             const  DmuErrHandler_f     err_handler,
                    pid_t               *child_pid,
                    DmuAllErrors_t      **errs)

The parameters of the `DmuCreateContext()` call are as follows:

`dmuctxt` |  | This parameter is returned with the address of the newly created API context. This parameter is passed to the API on all subsequent subroutine calls that require the program's API context.
`err_handler` |  | This parameter can be used to specify a user-defined error handling routine. The `DmuErrHandler_f` type is defined in `libdmfusr.h`. If the `err_handler` parameter is NULL, the default error handler, `DmuDefErrHandler` is used. For more information, see “DmuErrHandler_f”.
`child_pid` |  | This parameter specifies the pid of the child that is forked and executed to create the `dmusrcmd` process. This value is returned to the caller so that the caller is free to handle the termination of child signals as desired.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur.

If the `DmuCreateContext` call completes successfully, it returns `DmuNoError`.

#### `DmuDestroyContext` Subroutine

The `DmuDestroyContext` subroutine destroys the API context to which that `dmuctxt` points. The memory that had been allocated for its use is freed. The code is as follows:

    extern DmuError_t
    DmuDestroyContext(
                     void            *dmuctxt,
                     DmuAllErrors_t	 **errs)

The parameters of the `DmuDestroyContext()` call are as follows:

`dmuctxt` |  | This parameter is pointer to an API context that was previously created via `DmuCreateContext()`.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur.

### DMF File Request Subroutines

Each of the following subroutines makes a DMF file request. The context parameter that is included in each of these calls must have been already initialized via `DmuCreateContext`.

#### Copy File Requests

The `DmuCopyAsync` and `DmuCopySync` subroutines perform copy requests in the manner of the `dmcopy(1)` command. The code is as follows:

    extern DmuError_t
    DmuCopyAsync(
                  void            *dmuctxt,
            const char            *srcfile_path,
            const char            *dstfile_path,
            const int             copy_flags,
            const DmuCopyRanges_t *copyranges,
                  DmuReqid_t      *request_id,
                  DmuAllErrors_t  **errs)

    extern DmuError_t
    DmuCopySync(
                  void            *dmuctxt,
            const char            *srcfile_path,
            const char            *dstfile_path,
            const int             copy_flags,
            const DmuCopyRanges_t *copyranges,
                  DmuAllErrors_t  **errs)

The `DmuCopyAsync` subroutine returns immediately after the copy request has been forwarded to the `dmusrcmd` process. If a reply is desired, the caller must process the reply to this request.

The `DmuCopySync` subroutine does not return until the requested copy has either completed successfully or been aborted due to an error condition.

This request manipulates the destination file in exactly the same manner as that of the `to_file` argument of the `dmcopy` command.

The parameters of these routines are as follows:

`dmuctxt` |  | This parameter is a pointer to an API context that was previously created by `DmuCreateContext()`.
`srcfile_path` |  | This parameter specifies the pathname of the source (input) file for the copy operation. It must be an offline or dual state DMF file.
`dstfile_path` |  | This parameter specifies the pathname of the destination (output) file for the copy operation. This path must point to a file that exists or can be created in a DMF-managed filesystem that is native on the same host as that of the source file's filesystem.
`copy_flags` |  | This parameter specifies the OR'd value of the following copy operation flags as defined in `libdmfusr.h`: `COPY_PRESV_DFILE` - Do not truncate the destination file before the copy operation. `COPY_ADDR_ALIGN` - Allow an address in the destination file that is greater than the size of the file. `COPY_NOWAIT` - If the daemon is not available to process the request, do not wait. Return immediately.
`copyranges` |  | This parameter specifies a pointer to a `DmuCopyRanges_t` object, as defined in “DmuCopyRanges_t” and in `libdmfusr.h`. Currently, this object can have only one `DmuCopyRange_t` as defined in “DmuCopyRange_t” and in `libdmfusr.h`.
`request_id` |  | This parameter specifies a unique request ID. This value can be used when processing `DmuCompletion_t` objects to find the completion status.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur.

If the routine succeeds, it returns `DmuNoError`.

#### Fullstat Requests

The following routines send a `fullstat` request to the `dmusrcmd` process. The ultimate result of this request is the transfer of a `DmuFullstat_t` object to the caller. Code for the routines is as follows:

    extern DmuError_t
    DmuFullstatByPathAsync(
                             void           *dmuctxt,
                    const    char           *path,
                             DmuReqid_t     *request_id,
                             DmuAllErrors_t **errs)

    extern DmuError_t
    DmuFullstatByPathSync(
                             void           *dmuctxt,
                    const    char           *path,
                             DmuFullstat_t  **fullstatb,
                             DmuFhandle_t   **fhandle,
                             DmuAllErrors_t **errs)

    extern DmuError_t
    DmuFullstatByFhandleAsync(
                             void           *dmuctxt,
                    const    DmuFhandle_t   *client_fhandle,
                             DmuReqid_t     *request_id,
                             DmuAllErrors_t **errs)

    extern DmuError_t
    DmuFullstatByFhandleSync(
                             void           *dmuctxt,
                    const    DmuFhandle_t   *client_fhandle,
                             DmuFullstat_t  **fullstatb,
                             DmuAllErrors_t **errs)

The 'Sync' versions of these calls do not return until the `DmuFullstat_t` has been received or the request has been aborted due to errors.

The 'Async' versions of these routines return immediately after successfully forwarding the `fullstat` request to the `dmusrcmd` process. If a reply is desired, the caller must process the reply to this request . That is the only way to actually receive the `DmuFullstat_t` object, however. The `DmuFullstatCompletion` subroutine has been supplied to extract the `fullstat` information from a `fullstat` completion object.

The 'ByPath'versions of these calls allow the target file to be defined by its pathname.

The 'ByFhandle' versions of these calls allow the target file to be defined by its filesystem handle, the `fhandle`. These routines are valid only when the command making the call is on the DMF server machine, and they are valid only when a user has sufficient (root) privileges.

These routines can return a successful completion (`DmuNoError` ), but might still not return valid `DmuFullstat_t` information. The routines are designed to return the normal `stat` type information regardless of whether a DMAPI `fullstat` could be successfully completed. Upon return from these routines, the caller can use a macro defined in the `libdmfusr.h` file named `DMU_NO_FULLSTAT_INFO` with the address of the `DmuFullstat_t` block as the parameter and it will verify the validity of the DMAPI information in the `DmuFullstat_t` block.

The parameters of these routines are as follows:

`dmuctxt` |  | This parameter is a pointer to an API context that was previously created by `DmuCreateContext()`.
`path` |  | This parameter specifies the relative or absolute pathname of the target file.
`client_fhandle` |  | This parameter specifies the DMF filesystem `fhandle` of the target file.
`fullstatb` |  | This parameter specifies the pointer that will be returned with the `DmuFullstat_t fullstat` block.
`fhandle` |  | This parameter specifies a pointer that will be returned with the `DmuFhandle_t` value.
`request_id` |  | This parameter is set with the unique request ID of the `fullstat` request. You can use this value when processing `DmuCompletion_t` objects to find the request's completion status.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur.

If the routine succeeds, it returns `DmuNoError`.

#### Put File Requests

The following routines perform the put DMF request.

    extern DmuError_t
    DmuPutByFhandleAsync(
                           void            *dmuctxt,
                   const   DmuFhandle_t    *client_fhandle,
                   const   int             flags,
                   const   DmuByteRanges_t *byteranges,
                           DmuReqid_t      *request_id,
                           DmuAllErrors_t  **errs)

    extern DmuError_t
    DmuPutByFhandleSync(
                           void            *dmuctxt,
                   const   DmuFhandle_t    *client_fhandle,
                   const   DmuMigFlags_t   flags,
                   const   DmuByteRanges_t *byteranges,
                           DmuAllErrors_t  **errs)

    extern DmuError_t
    DmuPutByPathAsync(
                           void            *dmuctxt,
                   const   char            *path,
                   const   DmuMigFlags_t   flags,
                   const   DmuByteRanges_t *byteranges,
                           DmuReqid_t      *request_id,
                           DmuAllErrors_t  **errs)

    extern DmuError_t
    DmuPutByPathSync(
                           void            *dmuctxt,
                   const   char            *path,
                   const   DmuMigFlags_t   flags,
                   const   DmuByteRanges_t *byteranges,
                           DmuAllErrors_t  **errs)

The 'Sync' versions of these calls do not return until the put request has either completed successfully, or been aborted due to errors.

The 'Async' versions of these routines return immediately after successfully forwarding the put request to the `dmusrcmd` process. If a reply is desired, the caller must process the reply to this request.

The 'ByPath' versions of these calls allow the target file to be defined by its pathname.

The 'ByFhandle' versions of these calls allow the target file to be defined by its filesystem handle, the `fhandle`. These routines are valid only when the command making the call is on the DMF server machine, and they are valid only when a user has sufficient (root) privileges.

The parameters of these routines are as follows:

`dmuctxt` |  | This parameter is a pointer to an API context that was previously created by `DmuCreateContext()`.
`client_fhandle` |  | This parameter specifies the DMF filesystem `fhandle` of the target file. Valid for use only by a privileged (root) user.
`path` |  | This parameter specifies the relative or full path name of the target file.
`flags` |  | These parameters specify the following migration flags as defined in `libdmfusr.h`: `MIG_NONE` - No flags specified. `MIG_FREE` - Free the space associated with the file. `MIG_NOWAIT` - If the daemon is not available to process the request, do not wait. Return immediately.
`byteranges` |  | This parameter specifies a pointer to a `DmuByteRanges_t` object, as defined in `libdmfusr.h`. Currently, this object can have only one `DmuByteRange_t` as defined in `libdmfusr.h`.
`request_id` |  | This parameter specifies a unique request ID of the `put` request. This value can be used when processing `DmuCompletion_t` objects to find the request's completion status.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur.

If the routine succeeds, it returns `DmuNoError`.

#### Get File Requests

The following routines perform the get DMF request.

    extern DmuError_t
    DmuGetByFhandleAsync(
                           void             *dmuctxt,
                   const   DmuFhandle_t     *client_fhandle,
                   const   DmuRecallFlags_t flags,
                   const   DmuByteRanges_t  *byteranges,
                           DmuReqid_t       *request_id,
                           DmuAllErrors_t   **errs)

    extern DmuError_t
    DmuGetByFhandleSync(
                           void             *dmuctxt,
                   const   DmuFhandle_t     *client_fhandle,
                   const   DmuRecallFlags_t flags,
                   const   DmuByteRanges_t  *byteranges,
                           DmuAllErrors_t   **errs)

    extern DmuError_t
    DmuGetByPathAsync(
                           void             *dmuctxt,
                   const   char             *path,
                   const   DmuRecallFlags_t flags,
                   const   DmuByteRanges_t  *byteranges,
                           DmuReqid_t       *request_id,
                           DmuAllErrors_t   **errs)

    extern DmuError_t
    DmuGetByPathSync(
                           void             *dmuctxt,
                   const   char             *path,
                   const   DmuRecallFlags_t flags,
                   const   DmuByteRanges_t  *byteranges,
                           DmuAllErrors_t   **errs)

The 'Sync' versions of these calls do not return until the get request has either completed successfully, or has been aborted due to errors.

The 'Async' versions of these routines return immediately after successfully forwarding the get request to the `dmusrcmd` process. If a reply is desired, the caller must process the reply to this request.

The 'ByPath'versions of these calls allow the target file to be defined by its path name.

The 'ByFhandle' versions of these calls allow the target file to be defined by its filesystem handle, the `fhandle`. These routines are valid only when the command making the call is on the DMF server machine, and they are valid only when a user has sufficient (root) privileges.

The parameters of these routines are as follows:

`dmuctxt` |  | This parameter is a pointer to an API context that was previously created by `DmuCreateContext()`.
`client_fhandle` |  | This parameter specifies the DMF filesystem `fhandle` of the target file. Valid for use only by a privileged (root) user.
`path` |  | This parameter specifies the relative or full path name of the target file.
`flags` |  | These parameters specify the following recall flags as defined in `libdmfusr.h`: `RECALL_NONE` - No flags specified. `RECALL_NOWAIT` - If the daemon is not available to process the request, do not wait. Return immediately.
`byteranges` |  | This parameter specifies pointer to a `DmuByteRanges_t` object, as defined in `libdmfusr.h`. Currently, this object can have only one `DmuByteRange_t`, as defined in `libdmfusr.h`.
`request_id` |  | This parameter specifies a unique request ID of the `get` request. This value can be used when processing `DmuCompletion_t` objects to find the completion status.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur.

If the routine succeeds, it returns `DmuNoError`.

### Request Completion Subroutines

The request completion subroutines are provided so that the application can process the completion events of any asynchronous requests it might have issued. The caller can choose to process each request's completion object (`DmuCompletion_t`), or simply be notified when each request has responded with either an intermediate or final (completion) reply.

The asynchronous requests described previously along with the following completion subroutines allow the user to achieve maximum parallelization of the processing of all requests.

#### `DmuAwaitReplies` Subroutine

The `DmuAwaitReplies` subroutine performs a synchronous wait until the number of outstanding request replies of type type is less than or equal to `max_outstanding`. This subroutine is called by a user who does not want to perform individual processing of each outstanding request, but wants to know when a reply (intermediate or final) has been received for each request that has been sent to this point. Code for the routine is as follows:

    extern DmuError_t
    DmuAwaitReplies(
                    void            *dmuctxt,
                    DmuReplyType_t  type,
                    int             max_outstanding,
                    DmuAllErrors_t  **errs)

The parameters of this routine are as follows:

`dmuctxt` |  | This parameter is a pointer to an API context that was previously created by `DmuCreateContext()`.
`type` |  | This parameter defines the type of reply to be received. The caller can wait for an intermediate or final reply for the outstanding requests. See the definition of `DmuReplyType_t` in “DmuReplyType_t” or in `libdmfusr.h`.
`max_outstanding` |  | This parameter specifies the number of outstanding requests allowed for which the `type` reply has not been received before the subroutine returns. If this parameter is 0, all `type` replies will have been received when the routine returns.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur. Note that this error object refers to errors that occur while waiting and receiving the next reply.

If no errors occurred getting the next reply, this routine returns `DmuNoError`.

#### `DmuGetNextReply` Subroutine

The `DmuGetNextReply` subroutine returns the completion object of the next reply based on the order specified on the call.

The caller can specify `DmuIntermed` or `DmuFinal` for the `type` parameter. If `DmuIntermed` is specified and an intermediate reply is the next reply received and there are no completed replies available for processing, the `comp` parameter is not set (will be NULL) when the routine returns. An intermediate reply has no completion object associated with it, and a return of this type is informational only.

This subroutine performs a synchronous wait until a request reply of the type specified on the call is received. At the time of the call, any reply that has already been received and is queued for processing is returned immediately.

Code is as follows:

    extern DmuError_t
    DmuGetNextReply(
                    void            *dmuctxt,
                    DmuReplyOrder_t order,
                    DmuReplyType_t  type,
                    DmuCompletion_t **comp,
                    DmuAllErrors_t  **errs)

The parameters of this routine are as follows:

`dmuctxt` |  | This parameter is a pointer to an API context that was previously created by `DmuCreateContext()`.
`order` |  | This parameter defines the order in which the request replies should be returned. The caller can process the replies in the order the replies are received (`DmuAnyOrder`), or in the order the requests were issued (`DmuReqOrder`). See the definition of `DmuReplyOrder_t` in “DmuReplyOrder_t” or in `libdmfusr.h`.
`type` |  | This parameter defines the type of reply to be received. The caller can wait for an intermediate or final reply for the outstanding requests. The receipt of an intermediate reply returns no data.
`comp` |  | This parameter is set upon receipt of a final (completion) reply to the address of a completion object. The `reply_code` field of the `comp` parameter is the ultimate status of the request. A successful `comp` has a reply_code of `DmuNoError`. If the `reply_code` of `comp` is not `DmuNoError`, the `comp->allerrors` object will contain the error information needed to determine the cause of the error. Note that the `errs` parameter on the subroutine call does not contain the error information for the failed request.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur. Note that this error object refers to errors that occur while waiting and receiving the next reply. It does not refer to the errors that occurred during the request processing that is referenced by `comp`.

If no errors occurred getting the next reply, this routine returns `DmuNoError`. If there are no outstanding requests pending, a return code of `DME_DMU_QUEUEEMPTY` is returned. You can use a check for `DME_DMU_QUEUEEMPTY` to terminate a `while` loop based on this subroutine. Any other error return code indicates an error, and the `errs` parameter can be processed for the error information.

#### `DmuGetThisReply` Subroutine

The `DmuGetThisReply` subroutine returns the completion object of the specified request. This subroutine performs a synchronous wait until request reply specified on the call is received.

Code for this routine is as follows:

    extern DmuError_t
    DmuGetThisReply(
                    void            *dmuctxt,
                    DmuReqid_t      request_id,
                    DmuCompletion_t **comp,
                    DmuAllErrors_t  **errs)

The parameters of this routine are:

`dmuctxt` |  | This parameter is a pointer to an API context that was previously created by `DmuCreateContext()`.
`request_id` |  | This parameter is the unique request ID of the request for which the caller wants to wait.
`comp` |  | This parameter is set upon receipt of the final (completion) reply to the address of a completion object. The `reply_code` field of the `comp` parameter is the ultimate status of the request. A successful `comp` has a `reply_code` of `DmuNoError`. If the `reply_code` of `comp` is not `DmNoError`, the `comp->allerrors` object will contain the error information needed to determine the cause of the error. Note that the `errs` parameter on the subroutine call does not contain the error information for the failed request.
`errs` |  | This parameter is set with a pointer to a `DmuAllErrors_t` object if errors occur. Note that this error object refers to errors that occur while waiting and receiving this reply. It does not refer to the errors that occurred during the request processing that is referenced by `comp`.

If no errors occurred getting the next reply, this routine returns `DmuNoError`. Any other error return code indicates an error and the `errs` parameter can be processed for the error information.

#### `DmuFullstatCompletion` Subroutine

The `DmuFullstatCompletion` subroutine is supplied in the API to allow a user to make asynchronous `fullstat` requests and to ease the processing of the completion objects of those requests. When a `DmuCompletion_t` is returned to the caller via `DmuGetNextReply()` or `DmuGetThisReply()` , the user can extract the `DmuFullstat_t` and `DmuFhandle_t` information by calling this subroutine.

Code for the routine is as follows:

    extern DmuError_t
    DmuFullstatCompletion(
                    DmuCompletion_t  *comp;
                    DmuFullstat_t    **fullstatb,
                    DmuFhandle_t     **fhandle)

The parameters on this call are as follows:

`comp` |  | This parameter specifies the `DmuCompletion_t` object from an asynchronous `fullstat` request.
`fullstatb` |  | This parameter is returned with the `fullstat` information returned by the original request.
`fhandle` |  | This parameter is returned with the `fhandle` returned by the original request.

### Memory Management Subroutines

Memory management subroutines are available so that API users can efficiently manage their use of memory. Each subroutine defined in this section frees all of the memory associated with the object being deleted. It is safe to call all of these subroutines with a null object pointer.

The user should feel free to call any of these subroutines, using a parameter of the appropriate type that was used as input to one of the function or completion processing routines described previously.

- The following subroutine frees all memory associated with a `DmuAllErrors_t` object:

      extern void
      DmuDeleteAllErrors( DmuAllErrors_t *errs )

- The following subroutine frees all memory associated with a `DmuCompletion_t` object:

      extern void
      DmuDeleteCompletion( DmuCompletion_t *comp )

- The following subroutine frees all memory associated with a `DmuFullstat_t` object:

      extern void
      DmuDeleteFullstat( DmuFullstat_t *fullstat )

- The following subroutine frees all memory associated with a `DmuFhandle_t` object:

      extern void
      DmuDeleteFhandle( DmuFhandle_t *fhandle )

Prev | Table of Contents | Next
Appendix A. Messages |  | Appendix C. DMF Directory Structure Prior to Release 2.8