Chapter 6. Media-Specific Processes and Library Servers
Prev |  | Next

## Chapter 6. Media-Specific Processes and Library Servers

Media-specfic processes (MSPs) and library servers (LSs) migrate files from one media to another. There are the following types of MSPs:

- Tape MSP, which copies files from a disk to tape, or copies files from tape to disk.

- File transfer protocol (FTP) MSP, which allows the DMF daemon to manage data by moving it to a remote machine.

- Disk MSP, which migrates data to a directory that is accessible on the current systems and would be a cache disk if used in disk cache manager (DCM) mode.

LSs, like tape MSPs, copy files from a disk to a tape or from a tape to a disk. However, although the tape MSP and the LS have many characteristics in common, one of the primary differences is that while the tape MSP can manage at most one active copy of a migrated file, the LS can manage more than one copy. An LS is comprised of one or more volume groups. When a file is migrated from disk to tape, the selection policy can specify that it be copied to more than one volume group. Each volume group can manage at most one copy of a migrated file. Each volume group has an associated pool of tapes. Data from more than one volume group is never mixed on a tape.

This chapter discusses the following:

- “Tape MSP and LS Operations”

- “FTP MSP”

- “Disk MSP”

- “Disk MSP and Disk Cache Manager (DCM)”

- “Moving Migrated Data between MSPs and Volume Groups”

- “Converting from an IRIX DMF to a Linux DMF”

- “Converting from a Tape MSP to a Library Server”

- “Library Server Error Analysis and Avoidance”

- “Library Server Drive Scheduling”

- “Library Server Status Monitoring”

## Tape MSP and LS Operations

The tape MSP consists of the following programs:

- `dmatmsp`

- `dmatwc`

- `dmatrc`

The DMF daemon executes `dmatmsp` as a child process. The MSP communicates with the daemon through a pair of unnamed pipes. In turn, `dmatmsp` executes `dmatwc` (the write child) to write data to tape and `dmatrc` (the read child) to read data from tape.

The LS consists of the following programs:

- `dmatls`

- `dmatwc`

- `dmatrc`

The DMF daemon executes `dmatls` as a child process. In turn, `dmatls` executes `dmatwc` (the write child) to write data to tape and `dmatrc` (the read child) to read data from tape.

The `dmatmsp` or the `dmatls` program maintains the following types of records in its database:

- Catalog (CAT) records, which contain information about the files the MSP/LS maintains

- Volume (VOL) records, which contain information about the media the MSP/LS uses

The database is not a text file and cannot be updated by standard utility programs. Detailed information about the database and its associated utilities is provided in “CAT Database Records”, and “VOL Database Records”.

The tape MSP/LS provides a mechanism for copying active data from volumes that contain largely obsolete data to volumes that contain mostly active data. This process is referred to as volume merging or compression. Data on MSP/LS volumes becomes obsolete when users delete or modify their files. Volume merging can be configured to occur automatically (see “Configuring Maintenance Tasks for Tape MSP and LS” in Chapter 2). It can also be triggered by marking MSP/LS volumes as sparse with the `dmvoladm(8)` command.

The tape MSP/LS provides two utilities that read MSP/LS volumes directly:

- `dmatread(8)`, which copies all or part of a migrated file to disk

- `dmatsnf(8)`, which audits and verifies MSP/LS volumes

### Tape MSP/LS Directories

Each instance of the tape MSP/LS needs three types of directories, one for each of the following:

- Databases

- Database journal files

- Log files

Sites define the location of these directories by editing the base object configuration file parameters `HOME_DIR`, `JOURNAL_DIR`, and `SPOOL_DIR`, whose values are referred to as HOME_DIR, JOURNAL_DIR, and SPOOL_DIR in this document. A given instance of the tape MSP/LS creates a subdirectory named after itself in each of these three directories.

For example, if an instance of the tape MSP is called `cart1`, its database files reside in directory `HOME_DIR/cart1`. If another instance of the tape MSP is called `cart2`, its database files reside in `HOME_DIR/cart2`. If an instance of the LS is called `cart3`, its database files reside in `HOME_DIR/cart3`.

Similarly, MSP `cart1` stores its journal files in directory `JOURNAL_DIR/cart1` and its log files and other working files in `SPOOL_DIR /cart1.`

### Media Concepts

The tape MSP/LS takes full advantage of the capabilities of modern tape devices, including data compression and fast media positioning. To accommodate these capabilities and to provide recovery from surface or other media defects, `dmatmsp` and `dmatls` use a number of structural concepts built on top of traditional tape structure.

The components are as follows:

- The block is the basic structural component of most tape technologies. It is the physical unit of I/O to and from the media. The optimal block size varies with the device type. For example, the default block size for a 3480/3490 device is 65,536 bytes.

- A chunk is as much or as little of a user file as fits on the remainder of the tape (see Figure 6-1). Thus, every migrated file has at least one, and sometimes many, chunks. Such a concept is necessary because the capacity of a volume is unknown until written, both because of natural variation in the medium itself and because the effect of data compression varies with the data contents.

- A zone is a logical block containing several physical blocks ending with a tape mark. A zone has a target size that is configurable by media type. The default zone target size is 50 MB.

  The MSP or volume group writes chunks into the zone until one of three conditions occurs:

  - The zone size is exceeded

  - The MSP or volume group exhausts chunks to write

  - The end of tape is encountered

  Thus, the actual zone size can vary from well below the target size to the entire tape volume. A zone never spans physical volumes.

  The zone plays several roles:

  - The zone size is the amount of data that triggers `dmatmsp`/`dmatls` to start a process to write files to tape.

  - The MSP/LS records the position of the beginning of each zone in its database so that it can use fast hardware positioning functions to return there to restore the chunks in that zone.

  - When a tape volume develops a defect, the data loss usually will be restricted to the zone.

  Because getting the tape position and writing a tape mark can be very costly, the concept of a zone and the target size provides a way to control the trade offs between write performance, safety, and recall speed.

Figure 6-1 illustrates the way files are distributed over chunks, zones, and volumes, depending upon the file size. The tape with volume serial number (VSN) `VOL001` has two zones and contains six files and part of a seventh. The tapes with VSNs `VOL002` and `VOL003` contain the rest of file `g`. Notice that on `VOL001` file `g` is associated with chunk 7, while on the other two tapes it is associated with chunk 1. File `g` has three VSNs associated with it, and each tape associates the file with a chunk and zone unique to that tape.

Figure 6-1. Media Concepts

### CAT Database Records

Records in the tape catalog (CAT), `tpcrdm`, store the location of each file chunk in terms of its volume, zone, and chunk number. The key for these records is the file's bit file identifier (BFID).

You do not explicitly create CAT records in the database. They are created when files migrate.

The CAT portion of the MSP/LS database consists of the following files:

- `tpcrdm.dat`, which contains the data records themselves

- `tpcrdm.key1.keys` and `tpcrdm.key2.keys` , which contain the indexes to those records

For MSPs, the database definition file (in the same directory) that describes these files and their record structure is named `atmsp_db.dbd` . For LSs, it is named `libsrv_db.dbd`.

All files are non-ASCII and cannot be maintained by standard utility programs. The `dmcatadm` command provides facilities to create, query, and modify CAT database records (see “dmcatadm Command”).

| Note:: The ability to create or modify CAT database records with `dmcatadm` is provided primarily for testing purposes. In the normal course of operations, you would never use this capability.

### VOL Database Records

Records in the tape volume (VOL) portion of the MSP/LS database, `tpvrdm`, contain information about each volume that exists in the pool of tapes to be used by `dmatmsp` or `dmatls`. These records are indexed by the volume serial number (VSN) of each volume and contain such information as the volume's type, estimated capacity, label type, and a number of flags indicating the state of the volume. For LSs, the record also contains the volume's volume group or allocation group. Unlike the CAT records, you must create the VOL records in the database before using `dmatmsp` or `dmatls` for the first time.

The VOL portion of the MSP/LS database consists of two files:

- `tpvrdm.dat`, which contains the volume records themselves

- `tpvrdm.vsn.keys`, which contains the indexes to the records

For MSPs, the database definition file (in the same directory) that describes these files and their record structure is named `atmsp_db.dbd` . For LSs, it is named `libsrv_db.dbd`.

Both files contain binary data and require special maintenance utilities. The `dmvoladm` command, described in more detail in “dmvoladm Command”, provides facilities to create, query, and modify VOL records in the database. Additional database maintenance utilities are described in “Database Recovery” in Chapter 7.

| Note:: If you have more than one instance of the tape MSP or volume group, you must ensure that the volume sets for each are mutually exclusive.

### Tape MSP/LS Journals

Each instance of `dmatmsp` or `dmatls` protects its database by recording every transaction in a journal file. For MSPs, journal file pathnames have the following format:

    JOURNAL_DIR/msp_name/atmsp_db.yyyymmdd[.hhmmss]

For LSs, journal file pathnames have the following format:

    JOURNAL_DIR/ls_name/libsrv_db.yyyymmdd[.hhmmss]

The MSP/LS creates journal files automatically.

Existing journal files are closed and new ones created in two circumstances:

- When the first transaction after midnight occurs

- When the journal file reaches the size defined by the `JOURNAL_SIZE` configuration parameter

When the first transaction after midnight occurs, the existing open journal file is closed and the suffix `.235959` is appended to the current file name no matter what the time (or date) of closing. The closed file represents the last (or only) transaction log of the date yyyymmdd. A new journal file with the current date is then created.

When the journal file reaches `JOURNAL_SIZE`, the file is closed and the suffix .hhmmss is added to the name; hh, mm, and ss represent the hour, minute, and second of file closing. A new journal file with the same date but no time is then created.

For example, the following shows the contents of a `JOURNAL_DIR/msp_name` directory on 15 June 1998:

    atmsp_db.19980527.235959  atmsp_db.19980606.235959
    atmsp_db.19980528.235959  atmsp_db.19980607.235959
    atmsp_db.19980529.235959  atmsp_db.19980608.235959
    atmsp_db.19980530.235959  atmsp_db.19980609.235959
    atmsp_db.19980531.235959  atmsp_db.19980610.235959
    atmsp_db.19980601.235959  atmsp_db.19980611.235959
    atmsp_db.19980602.235959  atmsp_db.19980612.235959
    atmsp_db.19980603.235959  atmsp_db.19980613.235959
    atmsp_db.19980604.235959  atmsp_db.19980614.235959
    atmsp_db.19980605.235959  atmsp_db.19980615

For every date on which database transactions occurred, there will exist a file with that date and the suffix `.235959`, with the exception of an existing open journal file. Some dates may have additional files because the transaction log reached `JOURNAL_SIZE` at a specified time and the file was closed.

You can configure `daemon_tasks` parameters to remove old journal files (using the `run_remove_journals.sh` task and the `JOURNAL_RETENTION` parameter. For more information, see “Configuring Daemon Maintenance Tasks” in Chapter 2.

If an MSP/LS database becomes corrupt, recovery consists of applying the journal files to a backup copy of the database.

### Tape MSP/LS Logs

All DMF MSPs and LSs maintain log files named `msplog. yyyymmdd` in the MSP/LS spool directory which, by default, is `SPOOL_DIR/ mspname`. SPOOL_DIR is configured in the `base` object of the configuration file; mspname is the name of the MSP/LS in the `daemon` object of the configuration file; yyyymmdd is the current year, month, and day.

These log files are distinct from the logs maintained by the DMF daemon; however, some of the messages that occur in the daemon log are responses that the tape MSP/LS generates. The content of the log is controlled by the `MESSAGE_LEVEL` configuration parameter. For a description of the levels of logging available, see the `dmf.conf(5)` man page.

The `msplog.yyyymmdd` file is the primary log for the tape MSP/LS and contains most of the messages. This file is written by `dmatmsp`, `dmatls`, `dmatrc`, and `dmatwc`. A new `msplog. yyyymmdd` is created for each day.

This section describes informational statistics provided by the tape log files. These messages appear in the `SPOOL_DIR /msp_name/msplog. yymmdd` files. Timing information provided (such as MB transferred per second) should not be used as an accurate benchmark of actual data transfer rates. This information is provided for monitoring DMF and should only be used in comparison to similar data provided by DMF. Text in all uppercase references a parameter defined in the DMF configuration file. You can reference the comments in the sample configuration file or in the `dmf.conf(5)` man page for a more detailed definition of these parameters.

| Note: Because the MSP/LS will continue to create log files and journal files without limit, you must remove obsolete files periodically by configuring the `run_remove_logs.sh` and `run_remove_journals.sh` tasks in the configuration file, as described in “Configuring Daemon Maintenance Tasks” in Chapter 2.

Example 6-1. Tape MSP Statistics Messages

The following is an example of tape MSP statistics messages taken from an `msplog.yyyymmdd` file. These messages are automatically and periodically issued by the MSP.

    09:06:00-I 18979429-dmatmsp stats: children=3/3/0/4, btp=92274688/130740224/250609687, wc=2/4, cwc=0
    09:06:00-I 18979429-dmatmsp stats: data put=722.469 mb, data recalled= 45.089 mb
    09:06:00-I 18979429-dmatmsp stats:    Put_File -      6     33      0      6
    09:06:00-I 18979429-dmatmsp stats:    Get_File -      1     13      0      0
    09:06:00-I 18979429-dmatmsp stats: Delete_File -      0      1      0      0
    09:06:00-I 18979429-dmatmsp stats:  Cancel_Req -      0      6      0      0
    09:06:00-I 18979429-dmatmsp stats:    Flushall -      0      8      3      0
    09:06:00-I 18979429-dmatmsp stats:       Merge -     20     14      0      0
    09:06:00-I 18979429-dmatmsp stats: mc=4, ms=0, mu=0, sm=1

The information provided by these entries is defined as follows:

- `children=3/3/0/4` represents the total child processes (3), the active child processes (3), the clean processes running (0), and the configured value of `CHILD_MAXIMUM` (4). Clean children are used when a `dmatrc` or `dmatwc` process dies without cleaning up.

- `btp=92274688/130740224/250609687` represents the bytes queued for putting (92274688), the threshold at which to start the next put child (130740224), and the bytes assigned to socket I/O (250609687)

- `wc=2/4` represents the active write child processes (2) and the configured value of `MAX_PUT_CHILDREN` (4)

- `cwc=0` represents the process ID of the current write child (that is, the write child that is accepting data to write). 0 represents none.

The next line gives the total amount of data put (722.469 megabytes) and recalled (45.089 megabytes).

The next six lines provide statistics for each type of MSP request. Statistics information is provided only for requests that have been issued since the MSP was started. These lines have the following format:

    request_name      active      successful      errors      canceled

active represents the number of requests not yet completed; successful represents the number of successfully completed requests; error represents the number of requests that completed with errors; canceled represents the number of canceled requests.

The last line provides the following information:

- `mc` is the configured value for `MERGE_CUTOFF`, the cutoff to stop scheduling tapes for merging (4)

- `ms` is the configured value for `CACHE_SPACE`, the merge cache space available (0 bytes)

- `mu` is the merge cache space used (0 bytes)

- `sm` is the number of socket merge children (1)

Example 6-2. LS Statistics Messages

The following is an example of LS statistics messages taken from an `msplog.yyyymmdd` file. These messages are automatically and periodically issued by the LS.

    00:02:00-I 13902144-dmatls  vg9a16.stats: children=1/0/0/7, btp=28098297/0/0, wc=0/7, cwc=0
    00:02:00-I 13902144-dmatls  vg9a17.stats: children=1/0/0/7, btp=59032803/0/0, wc=0/7, cwc=0
    00:02:00-I 13902144-dmatls  vg9a16.stats: data put=608.607 mb, data recalled=114.270 mb
    00:02:00-I 13902144-dmatls  vg9a17.stats: data put=1068.423 mb, data recalled=210.575 mb
    00:02:01-I 13902144-dmatls  vg9a16.stats:    Put_File -     10    172      0     12
    00:02:01-I 13902144-dmatls  vg9a16.stats: Delete_File -      0    130      0      0
    00:02:01-I 13902144-dmatls  vg9a16.stats:  Cancel_Req -      0     12      0      0
    00:02:01-I 13902144-dmatls  vg9a16.stats:    Flushall -      0      2      0      0
    00:02:01-I 13902144-dmatls  vg9a16.stats:       Merge -     45     25      0     16
    00:02:01-I 13902144-dmatls  vg9a17.stats:    Put_File -     14    210      0      8
    00:02:01-I 13902144-dmatls  vg9a17.stats:    Get_File -      0      1      0      0
    00:02:01-I 13902144-dmatls  vg9a17.stats: Delete_File -      0    178      0      0
    00:02:01-I 13902144-dmatls  vg9a17.stats:  Cancel_Req -      0      8      0      0
    00:02:01-I 13902144-dmatls  vg9a17.stats:    Flushall -      0      2      0      0
    00:02:01-I 13902144-dmatls  vg9a17.stats:       Merge -     18     28      0     22
    00:02:01-I 13902144-dmatls  vg9a16.stats: mc=7, ms=500000000, mu=133107712, sm=0
    00:02:01-I 13902144-dmatls  vg9a17.stats: mc=7, ms=500000000, mu=73105408, sm=0

The fields in each message have the same meanings as in the MSP statistics messages (see Example 6-1), except that they are on a volume group basis.

The tape MSP/LS write child (`dmatwc`) and read child (`dmatrc`) also produce statistics messages in the MSP/LS log file. These messages contain timing statistics whose format changes from release to release, and they are not documented in this manual.

### Volume Merging

When users delete or modify their migrated files, the copy on tape becomes obsolete. Over time, some volumes will become entirely empty and can be reused. However, most volumes experience a gradual increase in the ratio of obsolete data to active data; such volumes are said to be sparsely populated or simply sparse. To reclaim the unused space on these volumes, DMF provides a volume merge facility, which copies the active data from several sparse volumes to a new volume, thus freeing the sparse volumes for reuse. Volume merging can be configured to occur automatically by using the `run_merge_tapes.sh` task (see “Configuring Maintenance Tasks for Tape MSP and LS” in Chapter 2).

Volume merging can also be done manually. `dmatmsp`/`dmatls` perform merge operations whenever sparse volumes and the necessary resources exist at the same time. Use the `dmvoladm select` directive to mark MSP or volume group volumes as sparse. (The `select` directive is described in “dmvoladm Command”.) Because the merge processing occurs simultaneously with other DMF activities, it is easiest to configure DMF to automatically perform merges at night or during other periods of relatively low activity.

The `dmatmsp/dmatls` utilities can perform volume-to-volume merging. Volume-to-volume merging is accomplished by moving data across a socket connection between the MSP/LS tape read-child and the MSP/LS tape write-child. The benefit of using a socket to transfer data between volumes is that you do not have to reserve disk space. The drawback to using a socket for data transfer is the cost of linking the process that performs the read with the process that performs the write.

In busy environments that have heavy contention for tape drives, the close coupling between the socket's tape reader and tape writer can be costly, especially when short files are being transferred. For large files, the overhead and possible delays in waiting for both tapes to be mounted is small compared to the benefit of rapid transfer and zero impact on free disk space. For this reason, you can move small files through a disk cache and big files through a socket. This process is mediated by the following configuration parameters:

Parameter | Description
`CACHE_SPACE` | Specifies the amount of disk space that will be used to temporarily store chunks during a merge operation.
`CACHE_DIR` | Specifies the directory into which the MSP/LS will store chunks while merging them from sparse tapes. If `CACHE_DIR` is not specified, `TMP_DIR` is used.
`MAX_CACHE_FILE` | Specifies the largest chunk that will be stored temporarily on disk during a merge operation.
`MERGE_CUTOFF` | Specifies the number of child processes after which the MSP or volume group will stop scheduling tapes for merging. This number is the sum of the active and queued children generated from gets, puts, and merges.

Using a small amount of disk space to hold small chunks can have a significant impact on the total time required to perform merges. The default configuration options are set to move 100% of merge data across sockets.

| Note: It is important to avoid volume merging on more than one MSP or volume group simultaneously if they share a tape device. If you initiate a merge process on more than one MSP or volume group on the same device at the same time (either by entering the same time in the DMF configuration file or by triggering the process manually), both processes will compete for tape transports. When a limited number of tape transports are available, a deadlock can occur. If you chose not to configure DMF to perform merges automatically by configuring the `run_tape_merge.sh` task, ensure that your `cron` jobs that automatically initiate volume merging refrain from initiating a second merge process until after all previously initiated merges are complete. You can accomplish this by using the `dmvoladm` command within the `cron` job to check for tapes that have the `hsparse` flag, as shown in the following example for MSPs:
    tapes=$(dmvoladm -m msp1 -c "count hsparse")
    if [[ -z "$tapes" ]]; then
            # start merge on msp2
            dmvoladm -m msp2 -c "select hfull threshold<=30"
    fi

### `dmcatadm` Command

The `dmcatadm(8)` command provides maintenance services for CAT records in the MSP/LS database.

When you are inside the `dmcatadm` interface (that is, when you see the `adm command_number >` prompt), the command has a 30-minute timeout associated with it. If you do not enter a response within 30 minutes of the prompt having been displayed, the `dmcatadm` session terminates with a descriptive message. This behavior on all the database administrative commands limits the amount of time that an administrator can lock the daemon and MSP/LS databases from updates.

| Note:: Most of these facilities, especially the ability to create and modify CAT database records, are intended primarily for testing purposes.

#### `dmcatadm` Directives

The `dmcatadm` command executes directives from `stdin` or from the command line when you use the `-c` option. All directives start with a directive name followed by one or more parameters. Parameters may be positional or keyword-value pairs, depending on the command. White space separates the directive name, keywords, and values.

The `dmcatadm` directives are as follows:

Directive | Description
`count` | Displays the number of records that match the expression provided.
`create` | Creates a CAT record.
`delete` | Deletes an existing CAT record.
`dump` | Prints the specified CAT records to standard out in ASCII; each database field is separated by the pipe character (`|`).
`help` | Displays help.
`list` | Shows the fields of selected CAT records. You may specify which fields are shown.
`load` | Applies records to the MSP/LS database obtained from running the `dump` directive.
`quit` | Stops program execution after flushing any changed database records to disk. The abbreviation `q` and the string `exit` produce the same effect.
`set` | Specifies the fields to be displayed in subsequent `list` directives.
`update` | Modifies an existing CAT record.
`verify` | Verifies the MSP/LS database against the `dmfdaemon` database.

The first parameter of most directives specifies the database records to manipulate, and the remaining parameters are keyword-value pairs.

The syntax for the `dmcatadm` directives is summarized as follows:

    count selection [limit]
    create key field...
    delete selection [limit]
    dump selection [limit]
    help
    list selection [limit] [format]
    load filename
    quit (or q or exit)
    set [format]
    update selection [limit] to fields...For MSPs:verify selection [entries] [mspname] [limit]For LSs:verify selection [entries] [vgnames] [limit]

The value for key may be a bit file identifier (BFID) designator in the form of a hexadecimal number.

The value for selection can be one of the following:

- A key or range of keys in the form key [`-`] [key]. `key-` specifies all records starting with key, and `-key` specifies all records up to key.

- The keyword `all`

- A period (`.`), which recalls the previous selection

- An expression involving any of the above, field value comparisons, `and`, `or`, or parentheses.

A field value comparison may use `<` (less than), `>` (greater than), `=` (equal to), `<=` (less than or equal to), or `>=` (greater than or equal to) to compare a field keyword to an appropriate value.

The syntax for selection is as follows:

    selection       ::=   or-expr
    or-expr        ::=   and-expr [ or or-expr  ]
    and-expr       ::=  nested-expr [ and and-expr  ]
    nested-expr     ::=   comparison | ( expression )
    comparison     ::=   key-range | field-keyword op  field-value
    op           ::=   < | > | = | >= | <=
    key-range      ::=   key [ - key] | [key - [key]] | key-macro
    key-macro     ::=   all
    field-keyword   ::=   name or abbreviation of the record field
    field-value     ::=   appropriate value for the field
    key          ::=   character representation of the record key

Thus valid selections could be any of the following:

    1510-1514
    10000000000-
    -15138
    all
    chunkoffset>0
    chunknumber>0 and writeage<5d
    . recordorder data
    vsn=S07638

#### `dmcatadm` Keywords

The field keywords listed below specify new values for fields. Some of the keywords are valid only if you also specify the `-u` option (unsafe mode), which allows changes to most database fields and supports the use of `dmcatadm` to repair database damage but must be used with care.

Keyword |  | Description
`chunkdata` (`cd`) |  | Specifies the actual number of bytes written to tape by the MSP or volume group for the chunk. In the case of sparse files, this field will be smaller than `chunklength`. This is valid only in unsafe (`-u`) mode.
`chunklength` (`cl`) |  | The size of the chunk in bytes; an integer. This is valid only in unsafe (`-u`) mode.
`chunknumber` (`cn`) |  | The ordinal of the chunk on its volume. For example, 1 if the chunk is the first chunk on the volume, 2 if it is the second, and so on. Valid only as part of selection.
`chunkoffset` (`co`) |  | The byte offset within the file where the chunk begins; an integer. For example, the first chunk of a file has `chunkoffset` 0. If that first chunk is 1,000,000 bytes long, the second chunk would have `chunkoffset` 1000000. This is valid only in unsafe (`-u`) mode.
`chunkpos` (`cp`) |  | The block offset within the zone where the chunk begins -- a hexadecimal integer. For example, the first chunk in a zone has `chunkpos` 1. A value of 0 means unknown. Valid only in unsafe (`-u`) mode in LS databases.
`filesize` (`fs`) |  | The original file size in bytes, an integer. This is valid only in unsafe (`-u`) mode.
`flags` (`fl`) |  | For future use.
`readage` (`ra`) |  | The date and time when the chunk was last read; the same as `readdate`, except specified as age.
`readcount` (`rc`) |  | The number of times the chunk has been recalled to disk; an integer.
`readdate` (`rd`) |  | The date and time when the chunk was last read, an integer that reflects raw UNIX time.
`volgrp` (`vg`) |  | The volume group name. This keyword is valid for LSs only.
`vsn` (`v`) |  | The volume serial numbers; a list of one or more 6-character alphanumeric volume serial numbers separated by colons (`:`).
`writeage` (`wa`) |  | The date and time when the chunk was written; the same as `writedate`, except specified as age . This is valid only in unsafe (`-u`) mode.
`writedate`(`wd`) |  | The date and time when the chunk was written, an integer that reflects raw UNIX time. This is valid only in unsafe (`-u` ) mode.
`zoneblockid` (`zb`) |  | Allows just the block ID portion of the `zonepos` to be displayed, returned, or changed. This is valid only in unsafe (`-u`) mode.
`zonenumber` (`zn`) |  | Allows just the zone number portion of the `zonepos` to be displayed, returned, or changed. This is valid only in unsafe (`-u`) mode.
`zonepos` (`zp`) |  | The physical address of the zone on the volume, expressed in the form `integer/ hexinteger`, designating a zone number and block ID. A value of zero is used for hexinteger if no block ID is known. integer is the same as `zonenumber`, and hexinteger is the same as `zoneblockid`. This is valid only in unsafe (`-u`) mode.

The date field keywords (`readdate` and `writedate`) have a value of either `now` or raw UNIX time (seconds since January 1, 1970). These keywords display their value as raw UNIX time. The value comparison `>` used with the date keywords means newer than the value given. For example, `>36000` is newer than 10AM on January 1, 1970, and `>852081200` is newer than 10AM on January 1, 1997.

The age field keywords (`readage` and `writeage`) let you express time as age, a string in a form such as `8w12d7h16m20s` (meaning 8 weeks, 12 days, 7 hours, 16 minutes, and 20 seconds old). The age keywords display their value as an integer followed by `w`, `d`, `h`, `m`, or `s` (weeks, days, hours, minutes, and seconds, respectively). The comparison `>` used with the age keywords means older than the value given (that is, `>5d` is older than 5 days).

The limit keywords limit the records acted upon:

Keyword |  | Description
`recordlimit` (`rl`) |  | Limits the number of records acted upon to the value that you specify; an integer.
`recordorder` (`ro`) |  | Specifies the order that records are scanned; may be `key`, `vsn`, or `data`. `key` specifies that records are scanned in ascending order of the chunk key. `vsn` specifies that records are scanned in ascending order of the chunk VSN. `data` specifies that records are scanned in the order in which they are stored in the database, which is fastest but essentially unordered.

The following keywords specify files of daemon database entries:

Keyword |  | Description
`entries` (`e`) |  | Specifies a file of daemon database entries; a string.
`mspname` (`mn`) |  | Specifies the name of the MSP associated with the record; a string.
`vgnames` (`vn`) |  | Specifies the names of the volume groups associated with the record; a quoted, space-separated list of names.

The format keyword selects a format to use for the display. If, for example, you want to display fields in a different order than the default or want to include fields that are not included in the default display, you specify them with the `format` keyword. Values for `format` can be `default`, `keyword`, or a list of field keywords enclosed in quotation marks.

For any field that takes a byte count, you may append the letter `k`, `m`, or `g` (in either uppercase or lowercase) to the integer to indicate that the value is to be multiplied by one thousand, one million, or one billion, respectively.

For information about the role of the `dmcatadm (8)` command in database recovery, see “Database Recovery” in Chapter 7.

Example 6-3. `dmcatadm` `list` directive

The following is sample output from the `dmcatadm` `list` directive. The file with key `3273d5420001e244` has two chunks because it spans two physical tape volumes; the first chunk contains bytes 0 through 24821759, and the second chunk bytes 24821760 (the `CHUNK OFFSET`) to the end of the file.

    adm 3>list 3273d5420001e242- recordlimit 10
                              WRITE     CHUNK       CHUNK   CHUNK
                        KEY     AGE    OFFSET      LENGTH     NUM       VSN
    -----------------------------------------------------------------------
           3273d5420001e242     61d         0    77863935      13    S12940
           3273d5420001e244     61d         0    24821760     168    S12936
           3273d5420001e244     61d  24821760    23543808       1    S12945
           3273d5420001e245     61d         0    51019776       2    S12945
           3273d5420001e246     61d         0    45629440      59    S12938
           3273d5420001e247     61d         0    35586048      60    S12938
           3273d5420001e248     61d         0     9568256       3    S12944
           3273d5420001e249     61d         0    14221312       4    S12944
           3273d5420001e24a     61d         0      458752       5    S12944
           3273d5420001e24b     61d         0    14155776       6    S12944

The following is sample output from the `dmcatadm` `list` directive for an LS. The file with key `3b4b28f2000000000000ae80` has 2 chunks because it was migrated to two different volume groups within this LS. The output from the `dmvoladm` `list` directive that follows shows that `VSN 000700` is assigned to the volume group named `vg8a15`, and `VSN 00727` is assigned to the volume group named `vg8a05` .

    # dmcatadm -m ls1
    adm 1>list 3b4b28f2000000000000ae80- recordlimit 4
                              WRITE       CHUNK       CHUNK CHUNK
                         KEY    AGE      OFFSET      LENGTH   NUM VSN
    --------------------------------------------------------------------
    3b4b28f2000000000000ae80     1d           0     2305938   120 000700
    3b4b28f2000000000000ae80     4d           0     2305938    32 000727
    3b4b28f2000000000000ae82     1d           0      234277   247 003171
    3b4b28f2000000000000ae82     1d           0      234277   186 003176

    adm 2> quit

    # dmvoladm -m ls1
    adm 1>list vsn=000700
                                                DATA    EOT   EOT             WR/FR
    VSN      VOLGRP LB      DATA LEFT        WRITTEN  CHUNK  ZONE   HFLAGS      AGE
    -------------------------------------------------------------------------------
    000700   vg8a15 al     150.280473     233.786093    123     9  ------u--     1d
    adm 2>list vsn=000727
                                                DATA    EOT   EOT             WR/FR
    VSN      VOLGRP LB      DATA LEFT        WRITTEN  CHUNK  ZONE   HFLAGS      AGE
    -------------------------------------------------------------------------------
    000727   vg8a05 al     159.107337     200.443980    102     6  ---------     1d

#### `dmcatadm` Text Field Order

The text field order for chunk records generated by the `dmdump(8)`, `dmdumpj(8)`, and the `dump` directive in `dmcatadm` is listed below. This is the format expected by the `load` directives in `dmcatadm`:

1. `C` (indicates the chunk record type)

2. `bfid` (hexadecimal digits)

3. `filesize`

4. `writedata`

5. `readdate`

6. `readcount`

7. `chunkoffset`

8. `chunklength`

9. `chunkdata`

10. `chunknumber`

11. `flags` (in octal)

12. `zoneposition` (`zonenumber`/`zoneblockid` ) (in hexadecimal)

13. `vsn`

14. `chunkpos` (in hexadecimal; only for LS)

### `dmvoladm` Command

The `dmvoladm(8)` command provides maintenance services for VOL records in the MSP/LS database. In addition to the creation and modification of volume records, `dmvoladm` has an important role in the recovery of VOL records from a database checkpoint and is the mechanism that triggers volume merge activity.

When you are inside the `dmvoladm` interface (that is, when you see the `adm command_number >` prompt), the command has a 30-minute timeout associated with it. If you do not enter a response within 30 minutes of the prompt having been displayed, the `dmvoladm` session terminates with a descriptive message. This behavior on all the database administrative commands limits the amount of time that an administrator can lock the daemon and MSP/LS databases from updates.

#### `dmvoladm` Directives

The `dmvoladm` command executes directives from `stdin` or from the command line when you use the `-c` option. The syntax is the same as for `dmcatadm`: a directive name followed by parameters or paired keywords and values, all separated by white space.

Directive | Description
`count` | Displays the number of records that match the expression provided.
`create` | Creates a VOL record.
`delete` | Deletes an existing VOL record.
`dump` | Prints the specified VOL records to standard output in ASCII. Each database field is separated by the pipe character (|).
`help` | Displays help.
`list` | Shows the fields of selected VOL records. You may specify which fields are shown.
`load` | Applies VOL records to the database obtained from running the `dump` directive.
`quit` | Stops program execution after flushing any changed database records to disk. The abbreviation `q` and the string `exit` produce the same effect.
`repair` | Causes `dmvoladm` to adjust the usage information for specified volumes based on CAT data in the database. This directive is valid only in unsafe (`-u`) mode.
`select` | Marks selected volumes as being sparse. Equivalent to `update expression to hsparse on`.
`set` | Specifies the fields to be shown in subsequent `list` directives.
`update` | Modifies an existing VOL record.
`verify` | Verifies the MSP databases against the `dmfdaemon` databases.

The syntax for the `dmvoladm` directives is summarized as follows:

    count selection
    create vsnlist [field...]
    delete selection [limit...]
    dump selection [limit...]
    help
    list selection [limit...] [format]
    load filename
    quit (or q, or exit)
    repair selection
    select selection [limit...]
    set [format]
    update selection [limit...] to field
    verify selection

The value for vsnlist may be a single 6-character volume serial number (VSN) or a range of VSNs separated by the hyphen (`-`) character. A VSN string may consists entirely of letters, entirely of digits, or may be a series of letters followed by digits. In a range of VSNs, the first must be lexically less than the second.

The value for selection may be one of the following:

- A vsnlist or range of VSNs in the form `vsn[-vsn ]`. `vsn-` specifies all records starting with vsn, and `-vsn` specifies all records up to vsn.

- A period (`.`), which recalls the previous selection

- The name of one of the flags in the keyword list that follows in this section.

- One of the words `all`, `used` , `empty`, or `partial` or any of the `hflags`, whose meanings are as follows:

  Flag | Description
  `all` | Specifies all volumes in the database
  `empty` | Specifies all volumes in which data written is 0
  `partial` | Specifies used volumes in which `hfull` is off
  `used` | Specifies all volumes in which data written is not 0

The syntax for selection is as follows:

    selection       ::=   or-expr
    or-expr        ::=   and-expr [ or or-expr  ]
    and-expr       ::=    nested-expr [ and and-expr  ]
    nested-expr     ::=   comparison | ( expression )
    comparison     ::=   vsnlist | field-keyword op  field-value
    op           ::=   < | > | = | >= | <=
    key-range      ::=   vsn [ - vsn] | [vsn - [vsn]] | key-macro
    key-macro      ::=   all | empty | used | partial | flag(s)
    field-keyword   ::=   name or abbreviation of the record field
    field-value     ::=   appropriate value for the field
    vsnlist        ::=   character representation of the volume serial number

Thus valid selections could be any of the following:

    tape01-tape02
    tape50-
    -vsn900
    all
    herr or hbadmnt
    used and hfull=off
    datawritten>0 and hfull=off
    . and eotchunk>3000 and (eotchunk<3500 or hfree=on)
    hfull and threshold<30

#### `dmvoladm` Keywords

The field keywords specify new values for fields:

Keyword |  | Description
`blocksize` (`bs`) |  | Specifies the data block size in bytes when the tape was first written; an integer. The default is 65,536. This keyword is used only when mounting tapes with existing valid data. When an empty tape is first written, the MSP or volume group uses the default value for the tape type, unless it is overridden by a value in the `BLOCK_SIZE` parameter for the tape device in the DMF configuration file. This is valid only in unsafe (`-u`) mode.
`chunksleft` (`cl`) |  | Specifies the number of active chunks on the volume; an integer. This is valid only in unsafe (`-u`) mode.
`dataleft` (`dl`) |  | Specifies the number of bytes of active data on the volume. You specify this number as an integer, but for readability purposes it is displayed in megabytes (MB). This is valid only in unsafe (`-u`) mode.
`datawritten` (`dw`) |  | Specifies the maximum number of bytes ever written to the volume. You specify this number as an integer, but for readability purposes it is displayed in MB. This is valid only in unsafe (`-u`) mode.
`eotblockid` (`eb`) |  | Specifies the blockid of the chunk containing the end-of-tape marker; a hexinteger. This is valid only in unsafe (`-u`) mode.
`eotchunk` (`ec`) |  | Specifies the number of the chunk containing the end-of-tape marker; an integer. This is valid only in unsafe (`-u`) mode.
`eotpos` (`ep`) |  | Specifies the absolute position of the end-of-tape marker zone in the form `integer/ hexinteger`, designating a zone number and block ID. A value of zero is used for hexinteger if no block ID is known. integer the same as `eotzone`, and hexinteger is the same as `eotblockid`. This is valid only in unsafe (`-u`) mode.
`eotzone` (`ez`) |  | Specifies the number of the zone containing the end-of-tape marker; an integer. This is valid only in unsafe (`-u`) mode.
`label` (`lb`) |  | Specifies the label type: `al` for ANSI standard labels; `sl` for IBM standard labels; or `nl` for nonlabeled volumes. The default is `al`.
`tapesize` (`ts`) |  | Specifies the estimated capacity in bytes; an integer. The default is 215 MB.
`threshold` (`th`) |  | Specifies the ratio of `dataleft` to `datawritten` as a percentage. This field cannot be displayed or updated.
`upage` (`ua`) |  | (Display only.) Specifies the date and time of the last update to the volume's database record. The same as for `update` , except that it is expressed as age. This is valid only in unsafe (`-u`) mode.
`update` (`ud`) |  | (Display only.) Specifies the date and time of the last update to the volume's database record, expressed as an integer that reflects raw UNIX time. This is valid only in unsafe (`-u`) mode.
`version` (`v`) |  | Specifies the DMF tape format version, an integer. This is valid only in unsafe (`-u`) mode.
`volgrp` (`vg`) |  | Specifies the volume group or allocation group. This field is valid only for LS databases.
`wfage` (`wa`) |  | Specifies the date and time that the volume was written to or freed for reuse. The same as for `wfdate`, except that it is expressed as age. This is valid only in unsafe (`-u`) mode.
`wfdate` (`wd`) |  | Specifies the date and time that the volume was written to or freed for reuse, expressed as an integer that reflects raw UNIX time. This is valid only in unsafe (`-u`) mode.

The date field keywords (`update` and `wfdate`) have a value of either `now` or raw UNIX time (seconds since January 1, 1970). These keywords display their value as raw UNIX time. The value comparison `>` used with the date keywords means newer than the value given. For example, `>36000` is newer than 10AM on January 1, 1970, and `>852081200` is newer than 10AM on January 1, 1997.

The age field keywords (`upage` and `wfage` ) let you express time as age, a string in a form such as `8w12d7h16m20s` (meaning 8 weeks, 12 days, 7 hours, 16 minutes, and 20 seconds old). The age keywords display their value as an integer followed by `w`, `d`, `h`, `m`, or `s` (weeks, days, hours, minutes, and seconds, respectively). The comparison `>` used with the age keywords means older than the value given (that is, `>5d` is older than 5 days).

The limit keywords restrict the volumes acted upon:

Keyword |  | Description
`datalimit` (no abbreviation) |  | Specifies a value in bytes. The directive stops when the sum of `dataleft` of the volumes processed so far exceeds this value.
`recordlimit` (`rl`) |  | Specifies a number of records; an integer. The directive stops when the number of volumes processed equals this value.
`recordorder` (`ro`) |  | Specifies the order that records are scanned; may be either `data` or `vsn`. `vsn` specifies that the records are scanned in ascending order of the chunk VSN. `data` specifies that the records are scanned in the order in which they are found in the database, which is fastest but essentially unordered.

The format keyword selects a format to use for the display. If, for example, you want to display fields in a different order than the default or want to include fields that are not included in the default display, you specify them with the `format` keyword. Values for `format` can be `default`, `keyword`, or a list of field keywords enclosed in quotation marks.

The flag keywords change the settings of hold flags:

Keyword |  | Description
`hbadmnt` (`hb`) |  | Indicates that the volume could not be mounted. If the problem causing the mount to fail is transient, the MSP will clear the flag the next time it attempts to mount the tape and succeeds. Typically this flag indicates a permanent condition that should be investigated and corrected. It is displayed as `--------b`. Currently, this flag is used only by MSPs.
`herr` (`he`) |  | Indicates that an I/O error has occurred on the volume; displayed as `e--------`. Currently, this flag is used only by MSPs.
`hflags` (no abbreviation) |  | (Display only.) Shows the complete set of hold flags as a 9-character string. Each flag has a specific position and alphabetic value. If the flag is off, a dash (`-`) is displayed in its position; if the flag is on, the alphabetic character is displayed in that position.
`hfree` (no abbreviation) |  | Indicates that the volume has no active data and is available for reuse after `HFREE_TIME` has expired, displayed as `-f-------`. See the `dmf.conf(5)` man page for information about the `HFREE_TIME` configuration parameter. This is valid only in unsafe (`-u`) mode.
`hfull` (`hu`) |  | Indicates that the volume cannot hold any more data; displayed as `------u--`. For LSs, this flag can also be set if error conditions indicate that no more data should be written to it.
`hlock` (`hl`) |  | Indicates that the tape cannot be used for either input or output. This is a transient condition; the flag will be cleared by the LS after a period of time has passed. Currently used only by the LS. Displayed as `----l----`.
`hoa` (`ho`) |  | Indicates that the volume is not to be used for either input or output, displayed as `--o------`.
`hro` (`hr`) |  | Indicates that the volume is read-only, displayed as `---r-----`; this inhibits the MSP from using the volume for output.
`hrsv` (`h*`) |  | Currently unused (reserved); displayed as `----*----` . This is valid only in unsafe (`-u`) mode.
`hsparse` (`hs`) |  | Indicates that the volume is considered sparse and thus a candidate for a volume merge operation, displayed as `-------s-` .

For any field that takes a byte count, you may append the letter `k`, `m`, or `g` (in either uppercase or lowercase) to the integer to indicate that the value is to be multiplied by one thousand, one million, or one billion, respectively.

For information about the role of the `dmvoladm` command in database recovery, see “Database Recovery” in Chapter 7. For details about `dmvoladm` syntax, see the man page.

Example 6-4. `dmvoladm` `list` directives

The following example illustrates the default format for the `list` directive when using an MSP. The column marked `HFLAGS` uses a format similar to the `ls` `-l` command in that each letter has an assigned position and its presence indicates that the flag is “on”. The positions spell the string `efor*lmusb` , representing `herr`, `hfree`, `hoa`, `hro`, `hrsv`, `hlock`, `hfull`, `hsparse`, and `hbadmnt`, respectively.

    adm 1>list S03232-S03254
                                         DATA    EOT             WR/FR
    VSN    LB      DATA LEFT        WRITTEN  CHUNK   HFLAGS      AGE
    ------------------------------------------------------------------
    S03232 sl      185.105446     400.000000     10 ------u--   997d
    S03233 sl      177.057792     400.000000      2 ------u--   495d
    S03234 sl      253.573185     400.000000    598 ------u--   906d
    S03235 sl      170.963133     400.000000     18 ------u--   497d
    S03236 sl      194.456616     400.000000     38 ------u--   915d
    S03237 sl      250.533926     400.000000     92 ------u--   803d
    S03238 sl        0.000000       0.000000      1 ---------   114d
    S03239 sl        0.000000       0.000000      1 ---------   114d
    S03240 sl        0.000000       0.000000      1 ---------   114d
    S03241 sl      252.162452     400.000000    325 ------u--   369d
    S03242 sl      166.635861     400.000000     81 ------u--   631d
    S03243 sl      202.468129     400.000000     26 ------u--   400d
    S03244 sl        0.000000       0.000000      1 ---------    96d
    S03245 sl      383.047890     400.000000     26 ------u--   212d
    S03246 sl      288.721920     400.000000      5 ------u--   687d
    S03247 sl      261.498716     400.000000    186 ------u--   691d
    S03248 sl      255.480486     400.000000     17 ------u--   288d
    S03249 sl      319.990661     400.000000    526 ------u--   253d
    S03250 sl        0.000000       0.000000      1 ---------   114d
    S03251 sl      241.785669     400.000000    533 ------u--   327d
    S03252 sl     1223.947545    1223.947545    157 ------u--    44d
    S03253 sl      386.038988     400.000000    636 ------u--   136d
    S03254 sl      170.798521     400.000000     38 ------u--   228d

The following example illustrates using the `list` command to show only volumes meeting some criterion (in this case, those having their `hfree` flag set):

    adm: 1>list hfree
                                         DATA    EOT             WR/FR
       VSN LB       DATA LEFT        WRITTEN  CHUNK   HFLAGS      AGE
    -------------------------------------------------------------------
    003249 sl         0.000000     115.000000      9 -f-r------     3h
    003250 sl         0.000000     115.000000      9 -f-r------     3h
    003251 sl         0.000000     115.000000     10 -f-r------     3h
    003252 sl         0.000000     115.000000     11 -f-r------     3h
    003255 sl         0.000000     115.000000     15 -f-r------     3h
    003258 sl         0.000000     115.000000     13 -f-r------     3h
    003263 sl         0.000000     115.000000     12 -f-r------     3h
    003264 sl         0.000000       0.000000      1 -f--------     4h
    003289 sl         0.000000       0.000000      1 -f-r------     3h
    003290 sl         0.000000     215.000000     29 -f-r------     3h
    003294 sl         0.000000       0.000000      1 -f--------     4h

The following example shows one way you can customize the list format to show only the fields that you want to see. The other way is to use the `set format` command with the same keyword list.

    adm 21>list S03232-S03254 format "eotchunk eotzone eotpos"
              EOT   EOT
    VSN     CHUNK  ZONE    EOTPOS
    -------------------------------
    S03232     10     2 2/4294967295
    S03233      2     2 2/4294967295
    S03234    598     2 2/4294967295
    S03235     18     2 2/4294967295
    S03236     38     2 2/4294967295
    S03237     92     2 2/4294967295
    S03238      1     1 1/4294967295
    S03239      1     1 1/4294967295
    S03240      1     1 1/4294967295
    S03241    325     2 2/4294967295
    S03242     81     2 2/4294967295
    S03243     26     2 2/4294967295
    S03244      1     1 1/4294967295
    S03245     26     2 2/4294967295
    S03246      5     2 2/4294967295
    S03247    186     2 2/4294967295
    S03248     17     2 2/4294967295
    S03249    526     2 2/4294967295
    S03250      1     1 1/4294967295
    S03251    533     2 2/4294967295
    S03252    157    17 17/2147483648
    S03253    636     2 2/4294967295
    S03254     38     2 2/4294967295

The following example gives a convenient way to show the several flag bits in a way different from their usual representation.

    adm 23>list 003232-003254 format "herr hfree hfull hlock hoa hro"
           herr hfree hfull hlock hoa hro
    VSN
    --------------------------------------
    003232  off   off    on    off off off
    003233  off   off   off    off off off
    003234  off   off   off    off off off
    003235  off   off   off    off off off
    003236  off   off    on    off off off
    003237  off   off    on    off off off
    003238  off   off    on    off off off
    003239  off   off    on    off off off
    003240  off   off   off    off off off
    003241  off   off    on    off off off
    003242  off   off    on    off off off
    003243  off   off   off    off off off
    003244  off   off   off    off off off
    003245  off   off    on    off off off
    003246  off   off   off    off off off
    003247  off   off    on    off off off
    003248  off   off    on    off off  on
    003249  off    on   off    off off  on
    003250  off    on   off    off off  on
    003251  off    on   off    off off  on
    003252  off    on   off    off off  on
    003253  off   off    on    off off  on
    003254  off   off    on    off off  on

The following example shows how to display only those tapes assigned to the volume group named `vg9a00`. This example is valid with LSs only.

    adm 3>list vg=vg9a00
                                                DATA    EOT   EOT             WR/FR
    VSN      VOLGRP LB      DATA LEFT        WRITTEN  CHUNK  ZONE   HFLAGS      AGE
    -------------------------------------------------------------------------------
    003210   vg9a00 al       1.048576       1.048576      3     2  ---------    11d
    003282   vg9a00 al      11.534336      11.534336     13     2  ---------     7d

#### `dmvoladm` Text Field Order

The text field order for volume records generated by the `dmdump(8)`, `dmdumpj(8)`, and the `dump` directive in `dmvoladm` is listed below. This is the format expected by the `load` directives in `dmvoladm`.

For MSP:

1. `V` (indicates the volume record type)

2. `vsn`

3. `lbtype`

4. `capacity`

5. `blocksize`

6. `hflags` (in octal)

7. `version`

8. `datawritten`

9. `eotchunk`

10. `eotposition` (`eotzone`/`eotblockid` ) (in hexadecimal)

11. `dataleft`

12. `chunksleft`

13. `wfdate`

14. `update`

15. `id` (in octal). This field indicates the type of process that last updated the record.

For LS:

1. `V` (indicates the volume record type)

2. `vsn`

3. `volgrp`

4. `lbtype`

5. `capacity`

6. `blocksize`

7. `hflags` (in octal)

8. `version`

9. `datawritten`

10. `eotchunk`

11. `eotposition` (`eotzone`/`eotblockid` ) (in hexadecimal)

12. `dataleft`

13. `chunksleft`

14. `wfdate`

15. `update`

16. `id` (in octal). This field indicates the type of process that last updated the record.

### `dmatread` Command

Use the `dmatread(8)` command to copy all or part of the data from a migrated file back to disk. You might want to do this if, for example, a user accidentally deleted a file and did not discover that the deletion had occurred until after the database entries had been removed by the hard delete procedure. Using backup copies of the databases from before the hard delete was performed, `dmatread` can restore the data to disk, assuming that the tape volume has not been reused in the meantime.

Example 6-5. Restoring Hard-deleted Files Using `dmatread`

To copy migrated files back to disk, perform the following steps:

1. Determine the BFID of the file you want to restore. You can use backup copies of `dmdlog` or your `dbrec.dat` files, or a restored dump copy of the deleted file's inode (and the `dmattr` command).

2. Using backup copies of the MSP/LS databases, use a `dmatread(8)` command similar to the following:

       dmatread -p /a/dmbackup -B 342984C50000000000084155

   `342984C50000000000084155` is the BFID of the file to be restored, and `/a/dmbackup` is the directory containing the backup copies of the MSP databases. Your file will be restored to the current directory as `B342984C50000000000084155`

DMF does not know the original name of the file; you must manually move the restored data to the appropriate file.

If you have access to chunk and VSN information for the file to be restored, you can use the `dmatread` `-c` and `-v` options and avoid using backup copies of the MSP/LS database. In this case, `dmatread` will issue messages indicating that the chunk is not found in the current database, but it will continue with the request and restore the file as described in this example.

### `dmatsnf` Command

Use the `dmatsnf(8)` command to verify the readability of or to audit the contents of MSP/LS volumes. You may also generate text database records that can be applied to the MSP/LS databases (using the `load` directive in `dmcatadm` and `dmvoladm`, respectively), in order to add the contents of a volume to the MSP/LS database (although this is impractical for large numbers of volumes).

`dmatsnf` can be used to verify one or more tape volumes against the MSP/LS databases. It also can be used to generate journal entries, which can be added to the MSP/LS databases by using the `load` directive in `dmvoladm` and `dmcatadm`.

### `dmaudit verifymsp` Command

Use the `verifymsp` option of the `dmaudit(8)` command to check the consistency of the DMF daemon and MSP/LS databases after an MSP, LS, DMF, or system failure. This command captures the database files and compares the contents of the daemon database with each MSP/LS database. Any problems are reported to standard output, but no attempt is made to repair them.

This function can also be done directly using `dmatvfy(8)` after a snapshot has been taken.

## FTP MSP

The FTP MSP allows the DMF daemon to manage data by moving it to a remote machine. Data is moved to and from the remote machine with the protocol described in RFC 959 (FTP). The remote machine must understand this specific protocol.

| Note:: It is desirable that the remote machine run an operating system based on UNIX, so that the MSP can create subdirectories to organize the offline data. However, this is not a requirement.

The FTP MSP does not need a private database to operate; all information necessary to retrieve offline files is kept in the daemon database, DMF configuration file, and login information file. The login information file contains configuration information, such as passwords, that must be kept private. As a safeguard, the MSP will not operate if the login information file is readable by anyone other than the system administrator.

### Processing of Requests

The FTP MSP is always waiting for requests to arrive from the DMF daemon, but, to improve efficiency, it holds `PUT` and `DELETE` requests briefly and groups similar requests together into a single FTP session. No `PUT` request will be held longer than 60 seconds. No `DELETE` request will be held longer than 5 seconds. `GET` requests are not held. The MSP will stop holding requests if it has a large amount of work to do (more than 1024 individual files or 8 MB of data). The FTP MSP also limits the number of FTP sessions that can be active at once and the rate at which new sessions can be initiated.

After a request has been held for the appropriate amount of time, it enters a ready state. Processing usually begins immediately, but may be delayed if resources are not available.

The following limits affect the maximum number of requests that can be processed:

- An administrator-controlled limit on the maximum number of concurrent FTP sessions per MSP (`CHILD_MAXIMUM`).

- An administrator-controlled limit on the number of child processes that are guaranteed to be available for processing delete requests (`GUARANTEED_DELETES`).

- An administrator-controlled limit on the number of child processes that are guaranteed to be available for processing `dmget(1)` requests (`GUARANTEED_GETS`).

- A system-imposed limit of 85 FTP sessions in any 60-second period. This limit is seldom a concern because of the MSP's ability to transfer many files in one session. Because requests are grouped into batches only when resources are immediately available, `GET` requests (which are not normally held) are batched when resources are in short supply.

Requests are processed by forking off a child process. The parent process immediately resumes waiting for requests to arrive from the DMF daemon. The child process attempts to initiate an FTP session on the remote FTP server. If the remote machine has multiple Internet Protocol (IP) addresses, all of them are tried before giving up. If the child process cannot connect, it waits 5 minutes and tries again until it succeeds.

Once a connection is established, the child process provides any required user name, password, account, and default directory information to the remote FTP server. `PUT`, `GET`, or `DELETE` operations are then performed as requested by the DMF daemon. `PUT`, `GET`, or `DELETE` operations are not intermixed within a batch. If an individual request does not complete successfully, it does not necessarily cause other requests in the same batch to fail. Binary transfer mode is used for all data transfer.

The stored files are not verbatim copies of the user files. They are stored using the same format used to write tapes, and you can use MSP utilities such as `dmatread` and `dmatsnf` to access the data in them.

### Activity Log

All DMF MSPs maintain log files named `msplog.yyyymmdd` in the MSP spool directory which, by default, is `SPOOL_DIR/ mspname`. SPOOL_DIR is configured in the `base` object of the configuration file; mspname is the name of the MSP in the `daemon` object of the configuration file; yyyymmdd is the current year, month, and day.

The activity log shows the arrival of new requests, the successful completion of requests, failed requests, creation and deletion of child processes, and all FTP transactions. Sensitive information (passwords and account information) does not appear in the activity log. In addition, the MSP lists the contents of its internal queues in its activity log if it is given an `INTERRUPT` signal.

| Note: Because the MSP will continue to create log files and journal files without limit, you must remove obsolete files periodically by configuring the `run_remove_logs` and `run_remove_journals` tasks in the configuration file, as described in “Configuring Daemon Maintenance Tasks” in Chapter 2.

### Messages

The MSP also recognizes and handles the following messages issued from the DMF daemon:

Message |  | Description
`CANCEL` |  | Issued when a previously requested action is no longer necessary, for example, when a file being migrated with a `PUT` request is removed. The MSP is able to cancel a request if it is being held or if it is waiting for resources. A request that has begun processing cannot be canceled and will run to normal completion.
`FINISH` |  | Issued during normal shutdown. When the MSP receives a `FINISH` message, it finishes all requested operations as quickly as it can and then exits.
`FLUSHALL` |  | Issued in response to the `dmdidle (8)` command. When the MSP receives a `FLUSHALL` message, it finishes all requested operations as quickly as it can.

| Caution: If the remote filesystem must be restored to a previous state, inconsistencies may arise: remote files that reappear after being deleted are never removed, and remote files that disappear unexpectedly result in data loss. There is presently no way to detect these inconsistencies. You should avoid situations that require the remote filesystem to be restored to a previous state.

## Disk MSP

The disk MSP (`dmdskmsp`) migrates data into a directory that is accessed on the current system. It uses POSIX file interfaces to open, read, write, and close files. The directory may be NFS-mounted, unless the disk MSP is configured as a disk cache manager (see “Disk MSP and Disk Cache Manager (DCM)”). The data is read and written with `root` (UID 0) privileges. By default, `dmdskmsp` stores the data in DMF-blocked format, which allows the MSP to do the following:

- Keep metadata with a file

- Keep sparse files sparse when they are recalled

- Verify that a file is intact on recall

The disk MSP does not need a private database to operate; all information necessary to retrieve offline files is kept in the daemon database and DMF configuration file.

The disk MSP may also be used as an import MSP. In this case, it only permits recalls and copies the data unchanged for a recall.

### Processing of Requests

The disk MSP is always waiting for requests to arrive from the DMF daemon, but, to improve efficiency, it holds `PUT` and `DELETE` requests briefly and groups similar requests together into a single session. No `PUT` request will be held longer than 60 seconds. No `DELETE` request will be held longer than 5 seconds. `GET` requests are not held. The MSP will stop holding requests if it has a large amount of work to do (more than 1024 individual files or 8 MB of data).

After a request has been held for the appropriate amount of time, it enters a ready state. Processing usually begins immediately, but may be delayed if resources are not available.

The following limits affect the maximum number of requests that can be processed:

- An administrator-controlled limit on the maximum number of concurrent operations per MSP (`CHILD_MAXIMUM`).

- An administrator-controlled limit on the number of child processes that are guaranteed to be available for processing delete requests (`GUARANTEED_DELETES`).

- An administrator-controlled limit on the number of child processes that are guaranteed to be available for processing `dmget(1)` requests (`GUARANTEED_GETS`).

Requests are processed by forking off a child process. The parent process immediately resumes waiting for requests to arrive from the DMF daemon.

`PUT`, `GET`, or `DELETE` operations are performed as requested by the DMF daemon. `PUT`, `GET`, or `DELETE` operations are not intermixed within a batch. If an individual request does not complete successfully, it does not necessarily cause other requests in the same batch to fail. Binary transfer mode is used for all data transfer.

The stored files are not verbatim copies of the user files. They are stored using the same format used to write tapes, and you can use MSP utilities such as `dmatread` and `dmatsnf` to access the data in them.

### Activity Log

All DMF MSPs maintain log files named `msplog.yyyymmdd` in the MSP spool directory which, by default, is `SPOOL_DIR/mspname` . SPOOL_DIR is configured in the `base` object of the configuration file; mspname is the name of the MSP in the `daemon` object of the configuration file; yyyymmdd is the current year, month, and day).

The log file shows the arrival of new requests, the successful completion of requests, failed requests, and creation and deletion of child processes. In addition, the MSP lists the contents of its internal queues in its activity log if it is given an `INTERRUPT` signal.

| Note: Because the MSP will continue to create log files and journal files without limit, you must remove obsolete files periodically by configuring the `run_remove_logs` and `run_remove_journals` tasks in the configuration file, as described in “Configuring Daemon Maintenance Tasks” in Chapter 2.

## Disk MSP and Disk Cache Manager (DCM)

The disk cache manager (DCM) lets you configure the disk MSP to manage data on secondary disk storage, allowing you to further migrate the data to tape as needed. The DCM provides an automated method of using secondary (slower and less-expensive) disk as a fast-access DMF cache for files whose activity levels remain high, while also providing migration to tape for those files requiring less frequent access.

To allow the disk store that is managed by the disk MSP to function as a dynamically managed cache (as opposed to a static store), DCM creates and maintains a filesystem attribute on each file that is created in the MSP `STORE_DIRECTORY`. This attribute is used by the `dmdskfree` process to evaluate files for downward migration and for possible removal from the disk cache. For this reason, the DCM `STORE_DIRECTORY` must be a local XFS or CXFS filesystem mount point with DMAPI enabled.

The DCM supports dual-resident state, in which files reside in the cache and also in a lower volume group. This provides the access speed of a disk file, but allows that cache file to be quickly released without the need to first write it to tape. This is directly analogous to the concept of a dual-state file in the standard DMF-managed filesystem.

Automated movement in the opposite direction (from tape back to the cache) is not available. Any recalls of files that no longer have copies held in the cache will come directly from tape; they are not recalled via the cache and they can only be restored to the cache by an explicit `dmmove(8)` command.

## Moving Migrated Data between MSPs and Volume Groups

DMF provides a mechanism to move copies of offline or dual-state files from one MSP or volume group to another. The `dmmove(8)` command takes a list of such files and moves them to a specified set of MSPs or volume groups. The list of MSPs or volume groups specified to the `dmmove` command indicates which MSPs or volume groups are to contain migrated copies of a file after the move process is completed. All other migrated copies are hard-deleted unless the `dmmove -d` option is used to select which copies are to be hard-deleted.

If a file's migrated state is offline, `dmmove` recalls the file to disk and then remigrates it to the specified MSPs or volume groups. (The one exception to this is that if a disk cache manager disk MSP copy exists, the file will be moved directly from that file copy.) When the migration process is complete, the online copy is removed. The file is recalled to a scratch filesystem that is specified by the `MOVE_FS` configuration parameter. If the file is dual-state, `dmmove` does not need to recall the file first, but instead uses the existing online copy.

The `dmselect(8)` command can be used to determine which files you want to move. `dmselect` selects files based on age, size, ownership, and MSP criteria. The output from the `dmselect` command can be used with the `dmmove` command. The `dmmove` command also accepts a list of pathnames as input.

See the man pages for `dmselect` and `dmmove` for all the possible options and further information.

## Converting from an IRIX DMF to a Linux DMF

You can convert IRIX DMF to Linux DMF and also convert Linux DMF to IRIX DMF . This section describes the necessary steps to convert an IRIX DMF to a Linux DMF.

DMF databases on IRIX machines cannot be copied to Linux machines because of binary incompatibility. Instead, they must be dumped to text on the IRIX machine, and the resulting text file must be loaded into the database on the Linux machine. DMF-managed filesystems, that is, filesystems containing user files that DMF has migrated, can be moved from an IRIX machine to a Linux machine.

It is assumed that sites converting DMF from an IRIX to a Linux machine (or vice versa) will obtain the help of SGI customer support; the following documentation is offered to familiarize you with the necessary steps. This procedure assumes the filesystems will be moved, and that this is done before the last step. It does not describe the steps required to move a filesystem.

Procedure 6-1. IRIX to Linux Conversion

1. Use `dmaudit` to verify that the DMF databases are valid. For more information, see the `dmaudit (8)` man page and the DMF Administrator's Guide for SGI InfiniteStorage and DMF Recovery and Troubleshooting Guide for SGI InfiniteStorage. To verify the databases that will actually be moved, you should change the filesystem migration levels in the `dmf.conf` file to `none`, run `dmdidle`, and then ascertain that all DMF activity has stopped before beginning this step. You should also use `dmsnap` to back up your databases.

2. Stop DMF on the IRIX system. If DMF is started again on the IRIX system during or after this procedure, the databases captured during step 3 might not reflect reality, and loss of data might result if you use them. To verify the consistency of the DMF databases, use the `dmdbcheck(8)` command.

3. Dump all of the DMF databases to text on the IRIX system. This should include the daemon database and the CAT and VOL databases for all tape MSPs and/or tape LSs. For more information, see the `dmdump(8)` man page.

4. Set up the `/etc/dmf/dmf.conf` file on the Linux system. Note that tape MSPs are not supported on Linux. If you do not have any tape MSPs, the conversion will be simpler if you name all of the FTP and DISK MSPs and the tape volume groups and LSs with the same names used on IRIX. This assumes that you do not already have MSPs or volume groups with these names on your Linux system.

   If you do change the name of an MSP or volume group, you must convert the daemon database. For more information on how to perform this conversion, see the documentation in the `dmconvertdaemon` script.

   If you do have a tape MSP, it must be converted to a volume group in an LS. Perhaps the easiest way to do this is to make the volume group name the same as the name of the MSP being converted. This method avoids making changes to the daemon database. For more information, see step 6. Use `dmcheck` to ensure that your new `/etc/dmf/dmf.conf` file is valid on the Linux system.

   Copy the text versions of the databases that you created in step 3 to the Linux machine.

5. If you have a tape MSP on the IRIX system, run the `dmconvertvol` script to convert the text version of its VOL database to the format required for a volume group in an LS. Also run the `dmconvertcat` script to convert the text version of its CAT database to the format required for a volume group in an LS. If the name of the volume group is different from the name of the tape MSP being converted, run the `dmconvertdaemon` script to convert the text version of the DMF daemon's database. The `dmconvertcat`, `dmconvertvol`, and `dmconvertdaemon` scripts reside in the `/usr/lib/dmf/support` file. Man pages do not exist for them, each script contains documentation.

6. Load the database files from the text files on the Linux machine. Use `dmdadm` to load the daemon database file. Use `dmcatadm` to load the CAT database for each of the tape LSs. Use `dmvoladm` to load the VOL database for each of the tape LSs. If you are converting multiple tape MSPs to multiple volume groups within a single LS, you must load each of their databases. See Example 6-6.

7. Use `dmdbcheck` to check the consistency of databases on the Linux machine.

8. Make sure all DMF filesystems are resident on the Linux machine.

9. Start DMF on the Linux machine and run `dmaudit`.

Example 6-6. IRIX to Linux Conversion (Single Tape LS)

In the following example, the IRIX machine has a single tape LS, named `ls1`, and no tape MSPs. The example assumes that the `/tmp/dmfdatabases` directory has been created, is initially empty, and contains enough space to accommodate the text versions of the databases. The example also assumes that the `HOME_DIR` configuration parameter is set to `/dmf/home` on both systems. After completing steps 1 and 2 of Procedure 6-1, the daemon database and the LS databases are dumped to text, as follows:

    $ dmdump -c /dmf/home/daemon > /tmp/dmfdatabases/daemon_txt
    $ dmdump /dmf/home/ls1/tpcrdm.dat > /tmp/dmfdatabases/ls1_cat_txt
    $ dmdump /dmf/home/ls1/tpvrdm.dat > /tmp/dmfdatabases/ls1_vol_txt

Next, the files in `/tmp/dmfdatabases` on the IRIX system are copied to `/tmp/dmftxtdb` on the Linux system. After creating the DMF configuration file on the Linux system, the databases are loaded on the Linux system, as follows:

    $ dmdadm -u -c "load /tmp/dmftxtdb/daemon_txt"
    $ dmcatadm -m ls1 -u -c "load /tmp/dmftxtdb/ls1_cat_txt"
    $ dmvoldadm -m ls1 -u -c "load /tmp/dmftxtdb/ls1_vol_txt"

Now `dmdbcheck` is run to verify the consistency of the databases, as follows:

    $ cd /dmf/home/daemon; dmdbcheck -a dmd_db
    $ cd /dmf/home/ls1; dmdbcheck -a libsrv_db

Example 6-7. IRIX to Linux Conversion (Two Tape MSPs)

In the following example, the IRIX machine has two tape MSPs. Their names are `msp1` and `msp2`. The example assumes that the `/tmp/dmfdatabases` directory has been created, is initially empty, and contains enough space to accommodate the text versions of the databases. This example also assumes that the `HOME_DIR` configuration parameter is set to `/dmf/home` on both systems. After completing steps 1 and 2 of Procedure 6-1, the daemon database and the tape MSP databases are dumped to text, as follows:

    $ dmdump -c /dmf/home/daemon > /tmp/dmfdatabases/daemon_txt
    $ dmdump /dmf/home/msp1/tpcrdm.dat > /tmp/dmfdatabases/msp1_cat_txt
    $ dmdump /dmf/home/msp1/tpvrdm.dat > /tmp/dmfdatabases/msp1_vol_txt
    $ dmdump /dmf/home/msp2/tpcrdm.dat > /tmp/dmfdatabases/msp2_cat_txt
    $ dmdump /dmf/home/msp2/tpvrdm.dat > /tmp/dmfdatabases/msp2_vol_txt

Next, we copy the files in `/tmp/dmfdatabases` on the IRIX system to `/tmp/dmftxtdb` on the Linux system. In this example, we assume that `msp1` will be converted to a volume group by the name of `vgpri` in LS `ls1`. Similarly, `msp2` will be converted to a volume group by the name of `vgsec` in LS `ls1`. Note that both of these volume groups will be in the same LS. To do this, you must ensure that the VSNs in each of the volume groups are unique.

After creating the `/etc/dmf/dmf.conf` file on the Linux system, the text versions of the database files are converted. First, the text versions of `msp1`'s files are converted to `vgpri`:

    $ dmconvertcat /tmp/dmftxtdb/msp1_cat_txt > /tmp/dmftxtdb/vgpri_cat_txt
    $ dmconvertvol /tmp/dmftxtdb/msp1_vol_txt vgpri > /tmp/dmftxtdb/vgpri_vol_txt

Next, the text versions of `msp2`'s files are converted to `vgsec`:

    $ dmconvertcat /tmp/dmftxtdb/msp2_cat_txt > /tmp/dmftxtdb/vgsec_cat_txt
    $ dmconvertvol /tmp/dmftxtdb/msp2_vol_txt vgsec > /tmp/dmftxtdb/vgsec_vol_txt

Since the name for the volume group has not been chosen to be the same as for the MSP being converted, the daemon database must be converted. For each tape MSP being converted to a volume group with a different name, `dmconvertdaemon` is run. In the following two steps, the first step handles the conversion of `msp1` to `vgpri` in the daemon database. The output of that command is then used as a parameter to `dmconvertdaemon` in the second step. The second step handles the conversion from `msp2` to `vgsec` in the daemon database.

    $ dmconvertdaemon msp1 vgpri /tmp/dmftxtdb/daemon_txt >\
    /tmp/dmftxtdb/daemon_pri_txt

    $ dmconvertdaemon msp2 vgsec /tmp/dmftxtdb/daemon_pri_txt > \
    /tmp/dmftxtdb/daemon_cnvt_txt

Next, the databases are loaded on the Linux system. Since two tape MSPs are being converted to volume groups within the same LS, two files are loaded into the `ls1` `CAT` database, and into the `ls1` `VOL` database:

    $ dmdadm -u -c "load /tmp/dmftxtdb/daemon_cnvt_txt"
    $ dmcatadm -m ls1 -u -c "load /tmp/dmftxtdb/vgpri_cat_txt"
    $ dmvoldadm -m ls1 -u -c "load /tmp/dmftxtdb/vgpri_vol_txt"
    $ dmcatadm -m ls1 -u -c "load /tmp/dmftxtdb/vgsec_cat_txt"
    $ dmvoldadm -m ls1 -u -c "load /tmp/dmftxtdb/vgsec_vol_txt"

Now `dmdbcheck` is run to verify the consistency of the databases.

    $ cd /dmf/home/daemon; dmdbcheck -a dmd_db
    $ cd /dmf/home/ls1; dmdbcheck -a libsrv_db

## Converting from a Tape MSP to a Library Server

For an existing MSP-based configuration to take advantage of the additional features of the LS, the existing databases must be converted. Several databases can be converted at the same time, or the conversion can be done in stages over a period of time. You can perform any of the following conversions:

- Convert just one MSP's databases to a new volume group in a new LS

- Convert an additional MSP to a new volume group within an existing LS

- Convert all databases at once

You can run a mixture of MSPs and LSs, with multiple copies of user files being held simultaneously by a volume group and an MSP. Procedure 6-2 provides the steps for conversion from tape MSP to LS databases.

Procedure 6-2. Tape MSP/LS Conversion

1. Run `dmcheck(8)`to check the existing configuration.

2. Copy the production configuration file (`/etc/dmf/dmf.conf` (2.8 or later) or `/etc/dmf/dmbase/host/ hostname/dmf_config`(2.7 or earlier)) and replace the definition of the MSPs to be converted with the stanzas defining the equivalent LS components to a new file, `/tmp/dmf.conf.new`. You might find it useful to examine the sample configuration to be found in `/usr/share/doc/dmf-version_number/info/sample/dmf.conf.ls` . Over time, many small changes have been made to benefit existing installations as well as new ones.

   To replace the definition of the MSPs, you must do the following:

   1. Delete the stanza for the MSP object.

   2. If there are no other references to the device object, remove it.

   3. Create an LS stanza and include the following parameters if they were specified in the MSP definition:

      - `CACHE_DIR`

      - `CACHE_SPACE`

      - `MAX_CACHE_FILE`

      - `MESSAGE_LEVEL`

      - `TASK_GROUPS`

   4. Replace the MSP's name in the `MSP_NAMES` (or `LS_NAMES`) directive in the daemon stanza with the name of this LS.

   5. Create a drive group stanza and include the following parameters if they were specified in the possibly-deleted device object:

      - `BLOCK_SIZE`

      - `LABEL_TYPE`

      - `MOUNT_SERVICE`

      - `MSG_DELAY`

      - `OV_ACCESS_MODES`

      - `OV_INTERCHANGE_MODES`

      - `POSITIONING`

      - `POSITION_RETRY`

      - `TMF_TMMNT_OPTIONS`

      - `VERIFY_POSITION`

      - `WRITE_CHECKSUM`

      Add a `MOUNT_SERVICE_GROUP` parameter to specify the TMF device group or OpenVault drive group. If `TMF_TMMNT_OPTIONS` contained a `-g` specification to provide this information, remove that part of it.

      The `DRIVE_GROUPS` parameter in the LS stanza should refer to this drive group.

   6. Create one volume group stanza per MSP being converted, possibly with the same names as the MSPs they are replacing, and include the following parameters if they were specified in the MSP definitions:

      - `HFREE_TIME`

      - `MAX_CHUNK_SIZE`

      - `MAX_PUT_CHILDREN`

      - `MERGE_CUTOFF`

      - `TIMEOUT_FLUSH`

      Include the `ZONE_SIZE` parameter from the possibly-deleted device object.

      The `VOLUME_GROUPS` parameter in the drive group stanza should refer to these volume groups. If their names differ from those of the MSPs they are replacing, update the `SELECT_MSP`/`SELECT_VG` policy parameters.

   7. In the task-group that controls filesystem backups with `run_full_dump.sh` and `run_partial_dump.sh` , change the `DUMP_DEVICE` parameter to refer to the drive group rather than to the possibly-deleted device object.

   To check this new configuration before placing it into production, before running `dmcheck(8)`, set the `DMF_CONFIG` environment variable to the absolute path of the file, as follows:

       setenv DMF_CONFIG /tmp/dmf.conf.new
       dmcheck

3. Run `dmaudit(8)` and `dmdbcheck(8)` to confirm that there are no problems with the current databases.

4. Stop DMF and put the new configuration in place. You can run `dmcheck(8)` again, if you wish.

5. Run `/usr/lib/dmf/support/dmmsptols`, as described in the man page. This process might take some time (even several hours for a large configuration). The selection of MSPs that must be converted at this point is determined by the changes made to the configuration in the previous step.

   If there are any problems, the `dmmsptols` process will instruct you how to back out of the conversion by using the backups it created.

6. Start DMF and run `dmaudit(8)` and `dmdbcheck(8)`.

7. At a later time, you can make additional changes to the configuration to enable or configure new features, such as allocation groups, error recovery, or resource schedulers and watchers. The parameters controlling these are described in Chapter 2, “Configuring DMF”, or elsewhere in this chapter.

When this procedure is followed, the resulting configuration will schedule tape merges for all volume groups to be done at once. This does not cause problems for the LS as it would for the MSP-based configuration, but you might wish to have finer control over the process. You can do this by creating new task-group objects just to control tape merging, and invoking them with a `TASK_GROUPS` parameter inside the volume groups' stanzas. The other parameters inside the original task-group should still be invoked only from the LS's stanza.

Alternatively, the `RUN_TASK` parameters can be placed directly in the volume group stanza; they can be specified outside a task-group's stanza.

The improved tape positioning code specified by the drive group's `POSITIONING` parameter to `data` will be activated only for data written by the volume group (that is, newly migrated files and files that have been merged from a tape written by the MSP to a new volume group one). The improvement in performance will become more noticeable over time, as a greater proportion of data fits into these categories.

## Library Server Error Analysis and Avoidance

Unlike the MSP, the drive group component of the LS monitors tape use, analyzing any failures, and using this information to avoid future errors.

The drive group component can react to some failures without looking for any patterns of behavior. Among these are the following:

- Mounting service failure. If the mounting service is TMF, by default, DMF makes one attempt to restart it. If this attempt does not succeed, DMF notifies the administrator by e-mail and waits for the administrator's intervention. When TMF is back again, DMF resets the auto-restart flag so that if TMF fails again, it will once again make one attempt to restart it.

  If OpenVault is the mounting service, by default, no attempt is made to restart it. Instead, an e-mail is sent to the administrator.

  A site can set the number of automatic restart attempts by using the drive group's `MAX_MS_RESTARTS` configuration parameter, but caution and thorough testing are advised. There are many possible failure modes for a mounting service, and automated restarts might not always be appropriate.

- Tape volume is not in the tape library. Obviously, this problem will not be fixed by trying again. To prevent further access, the volume is locked by setting the `HLOCK` flag, as described below, and the user requests that triggered the access attempt are retried on another tape, if possible; otherwise, they are aborted. The administrator is notified by e-mail.

- For TMF only, a tape mount was cancelled by an operator or administrator. Although the user requests are retried or aborted, the volume is not disabled. If the volume were disabled, it would be inaccessible for a period of time (default 24 hours) unless `dmvoladm` were used to preempt this delay. All operators do not necessarily have access to the `dmvoladm` command.

  Because the reason for the cancellation is unknown to DMF, repeated requests for the same volume are quite possible, and the operator might have to cancel each one.

The drive group handles other types of failure by examining the recent history of the tape volume and the tape drive that was used. The drive group maintains records of past tape I/O errors, and uses these to control the way it reacts to future errors.

For example, if a tape has been unusable several times in a row, even though different tape drives were used, the drive group concludes that the problem most likely involves the tape volume rather than the drive. Therefore, it suspends use of that tape for a while, forcing DMF to migrate to a different tape in that volume group, or to recall the file from another tape held by a different volume group. This suspension is usually done by setting the `HLOCK` flag in the tape's entry in the volume database. This makes the tape inaccessible to the volume group for both reading and writing until it is automatically cleared after `REINSTATE_VOLUME_DELAY` minutes.

If a variety of volumes fail on a specific drive but are usable on other drives, a drive problem is likely, and the tape drive can be automatically configured down if permitted by the administrator's setting of `DRIVES_TO_DOWN` to a value higher than its default of zero. When a drive is configured down in this way, it is configured up again after `REINSTATE_DRIVE_DELAY` minutes.

The analyses of drive and volume errors are performed independently of each other; it is possible for one additional error to result in both the drive and the volume being disabled.

There are several reasons for reinstating drives and volumes after a delay. The most important is that the analyses of previous failures might lead to a faulty conclusion in some situations, such as when DMF is under a very light load, or when multiple failures occur concurrently. A wrong diagnosis might impact DMF's performance, and should not be accepted indefinitely. Disabling a suspected drive or volume for a while is usually enough to break any repetitive cycles of failure. If such patterns re-establish themselves when the reinstatement occurs, the drive group will again analyze the behavior, possibly reaching a different conclusion, and again try to prevent it.

There are some variations from these general reactions. For example, if a tape volume with existing data on it is diagnosed as faulty when appending new data, instead of setting the `HLOCK` flag, the drive group sets `HFULL`, which results in the tape being used in a read-only mode until eventually emptied by merges or hard deletion of its files. `HFREE_TIME` seconds after it becomes empty, it may be placed back into use unless the administrator has decided, possibly as a result of testing it, that it should be deleted or replaced.

In all of these situations, the administrator is notified by e-mail.

If it is considered desirable to return a volume or drive to service earlier than defined in the DMF configuration, the appropriate command (`dmvoladm`, `tmconfig`, or `ov_drive`) can be safely used.

## Library Server Drive Scheduling

When multiple volume groups are requesting the use of more tape drives than exist in the drive group, the resource scheduler is used to decide which volume groups should wait, and which should be assigned the use of the drives.

The resource scheduler is unaware of non-volume-group activity on the drives in its drive group. Such activity includes MSPs, XFS dumps, and any direct tape use by the system's users, and does not prevent the LS from working properly, though it might be less than optimal.

By default, the resource scheduler uses a round-robin based algorithm, but a site can assign different weightings to different volume groups to meet local requirements. (For more information, see “Resource Scheduler Objects” in Chapter 2).

Some sites will have requirements that cannot be met by a general purpose algorithm. Such sites can write their own resource scheduler algorithms in C++, to be used in place of the supplied one. Instructions can be found in the `/usr/share/doc/dmf-version_number/info/sample/RSA.readme` file.

## Library Server Status Monitoring

You can observe the performance of the LS in two ways. You can monitor its log file with a tool like `tail -f`, which allows an experienced administrator to follow the flow of events as they happen. You can also use the resource watcher component, when enabled by use of the `WATCHER` parameter in the `libraryserver` configuration stanza.

The resource watcher is intended to give the administrator a view of the status of an LS and some of its components. It maintains a set of text files on disk, which are rewritten as events happen. These files can be found in the `SPOOL_DIR/lsname/_rwname` directory, where `SPOOL_DIR` is defined in the DMF configuration file, as are the names of the LS and resource watcher (`lsname` and `rwname` in the following example). The easiest way to find the precise path is to look in the LS log file for messages like the following:

    dmatls rwname.config_changed:
    Resource Watcher output files will be placed in /dmf/spool/lsname/_rwname at DMF startup or
    whenever the configuration file is altered or "touch"ed.

This message is issued at DMF startup or whenever the configuration file is altered or its modification time changes (for example, by using the `touch(1)` command.

The `SPOOL_DIR/lsname/_rwname` directory contains files with names ending in `.html`, which are automatically refreshing HTML files. You can access these files by using a browser running on the same machine. The following example shows an LS page that contains links to drive group pages, and they in turn have links to volume group pages, if the volume groups are active at the time.

    netscape file:/dmf/spool/lsname/_rwname/lsname.html

If running the browser on the DMF machine is inconvenient, you can include the directory in your HTTP server configuration to allow those same pages to be accessed via the web.

This directory also contains files whose names end in `.txt`, designed to be parsed with programs like awk. The data format is described by comments within those files and can be compared with the equivalent HTML files. If the format of the text ever changes, the version number will change. If the changes are incompatible with previous usage, the number before the decimal point is altered. If they are compatible, the number after the decimal point is altered. An example of compatibility is adding extra fields to the end of existing lines or adding new lines. Programs using these files should check the version number to ensure compatibility. Also, it might be useful to check the DMF version shown by `dmversion(1)` and the IRIX version from `uname(1)`.

Prev | Table of Contents | Next
Chapter 5. The DMF Lock Manager |  | Chapter 7. DMF Maintenance and Recovery