Chapter 1. Introduction
Prev |  | Next

## Chapter 1. Introduction

This chapter provides an overview of the Data Migration Facility (DMF) and its administration. It discusses the following:

- “What Is DMF?”

- “How DMF Works”

- “Ensuring Data Integrity”

- “DMF Architecture”

- “Capacity and Overhead”

- “DMF Administration”

- “The User's View of DMF”

- “DMF File Concepts and Terms”

- “Command Overview”

## What Is DMF?

DMF is a hierarchical storage management system for SGI environments. DMF allows you to oversubscribe your online disk in a manner that is transparent to users; a user cannot determine, by using POSIX-compliant commands for filesystem enquiry, whether a file is online or offline. Only when special commands or command options are used can a file's actual residence be determined. This transparent migration is possible because DMF leaves inodes and directories intact within the native filesystem.

DMF automatically detects a drop below the filesystem free-space threshold and migrates selected data from expensive online disk to cheaper secondary storage, such as tapes. DMF automatically recalls the file data from offline media when the user accesses the file with normal operating system commands.

Figure 1-1 provides a conceptual overview of the data flow between applications and storage media.

Figure 1-1. Application Data Flow

DMF supports a range of storage management applications. In some environments, DMF is used strictly to manage highly stressed online disk resources. In other environments, it is also used as an organizational tool for safely managing large volumes of offline data. In all environments, DMF scales to the storage application and to the characteristics of the available storage devices.

DMF interoperates with the following:

- Standard data export services such as Network File System (NFS) and File Transfer Protocol (FTP)

- XFS filesystems

- CXFS (clustered XFS) filesystems

- Microsoft's Server Message Block (SMB), which is also known as the Common Internet File System (CIFS), as used by Samba when fileserving to Windows systems

By combining these services with DMF you can configure an SGI system as a high-performance file server.

DMF transports large volumes of data on behalf of many users. Because system interrupts and occasional storage device failures cannot be avoided, it is essential that the safety and integrity of data be verifiable. Therefore, DMF also provides tools necessary to validate your storage environment.

DMF has evolved around these customer requirements for scalability and the safety of data. As a filesystem migrator, DMF manages the capacity of online disk resources by transparently moving file data from disk to offline media. Most commonly, the secondary storage is tape, managed by OpenVault or the Tape Management Facility (TMF). However, the secondary storage can be any bulk-storage device accessible locally through NFS or FTP.

### DMF Client and Server

DMF includes the following software subsystems:

- Server, which provides the full set of DMF functionality, including the DMF daemon, infrastructure, user and administrator commands, online manuals, and all man pages. This applies to IRIX systems and SGI ProPack for Linux 64-bit on SGI Altix 3000 systems.

- Client, which provides the limited set of user commands, libraries, and a subset of the man pages. This applies to all supported operating systems (see “Hardware and Software Requirements”).

Only one of these subsystems can be installed on a given machine.

### Hardware and Software Requirements

The DMF server subsystem runs on the following:

- SGI ProPack 2.2.1 or later for Linux on SGI Altix 3000 systems

- IRIX 6.5.2 or later

The DMF client subsystem supports nodes running the following operating systems:

- SGI ProPack 2.2.1 or later for Linux

- IRIX 6.5.2 or later

- IBM AIX 5L version 5.1 ML 4 (64-bit kernel mode) APAR number IY42428

- Red Hat Linux on supported IA32 platforms:

  - Red Hat Linux 7.3

  - Red Hat Linux 8.0

  - Red Hat Linux 9

- Sun Microsystems Solaris:

  - Solaris 8 plus appropriate patch (see the release notes)

  - Solaris 9 plus appropriate patch

- Microsoft Windows:

  - Windows 2000 Service Pack 3 or Service Pack 4

  - Windows XP Service Pack 1

The client-only user commands are as follows:

    dmattr
    dmcopy
    dmget
    dmfind
    dmls
    dmput

The DMF `libdmfusr.so` user library lets you write your own custom DMF user commands that use the same application program interface (API) as the above DMF user commands.

The most commonly used devices on IRIX and Linux systems are DLT 4000/7000, SCSI versions of IBM 3590, and STK TimberLine and RedWood drives. All STK robots, Grau, and IBM 3494 are supported.

| Note: On operating systems other than IRIX, the DMF client commands rely on the `xinetd` daemon to communicate with the DMF server machine. That communication is based on the `TCPMUX` functionality in `xinted`, which is not present in versions of `xinetd` prior to `xinetd`-2.3.11. If you want to export DMF-managed filesystems, the machine doing the exporting must run the appropriate level of `xinetd`. The following releases contain the required version of `xinetd` and are supported for exporting DMF-managed filesystems: `xinetd` version 2.3.11 available with SGI ProPack 2.3 `xinetd` 2.3.10 in Solaris

### DMAPI Requirement

For filesystems to be managed by DMF, they must be mounted on the DMF server in order to enable the Data Management (DMAPI) interface. Do one of the following for each platform:

- IRIX:

  - Use the following command:

        mount -o dmi

  - Declare parameter 4 in the `fstab` entry to be `dmi`

- Linux:

  - Use the following command:

        mount -o dmapi -o mtpt = mountpoint

  - Add `dmapi, mtpt = mountpoint` to the fourth field in the `fstab` entry

For more information, see the `mount` and `fstab` man pages.

### Licensing Requirement

The software licensing used by DMF servers is based on the FLEXlm product from Macrovision Corporation. You must have a separate license for each DMF server. (No licensing is required on the client host.)

Software keys are used to enforce licensing. DMF licenses apply to a single specific system. DMF license fees vary depending on the amount of data being managed.

When you order DMF, you will receive an entitlement ID and the URL to a key generation webpage. You must submit the system host ID, hostname, and entitlement ID when requesting your permanent DMF license. To determine the hostname and host ID number: launch `dmmaint` and select License Info.

To obtain your permanent DMF license, follow the instructions on the key generation page. After the required information is provided, a key will be generated and displayed on the webpage along with installation instructions. You can use the Update License button on the `dmmaint` GUI to install the license.

For more information about licensing, see the following webpage:

http://www.sgi.com/support/licensing

## How DMF Works

As a DMF administrator, you determine how disk space capacity is handled by selecting which filesystems DMF will manage and by specifying the volume of free space that will be maintained on each filesystem. Space management begins with a list of user files that are ranked according to criteria you define. File size and file age are among the most common ranking criteria.

File migration occurs in two stages:

- Stage One: A file is copied (migrated) to secondary storage.

- Stage Two: After the copy is secure, the file is eligible to have its data blocks released (this usually occurs only after a minimum space threshold is reached).

A file with all offline copies completed is called fully migrated. A file that is fully migrated but whose data blocks have not yet been released is called a dual-state file; its data exists both online and offline, simultaneously. After a file's data blocks have been released, the file is called an offline file.

You choose both the percentage of filesystem volume to migrate and the volume of free space. You can trigger file migration, or file owners can issue manual migration requests.

Offline media is the destination of all migrated data and is managed by daemon-like DMF components called the media-specific process (MSP) and the library server (LS).

| Note: Linux systems do not support the tape MSP. Tape support on Linux is available only via the LS.

The following types of MSPs are supported:

- Disk

- FTP

- MSP

In addition, you can configure the disk MSP to run in disk cache manager (DCM) mode for n-tier capability. DMF can manage the disk MSP's storage filesystem and further migrate it to tape, thereby using a slower and less-expensive dedicated filesystem as a cache to improve the performance when recalling files.

The FTP MSP (`dmftpmsp`) uses the FTP protocol to transfer to and from disks of another system on the network. The disk MSP (`dmdskmsp`) is similar, but uses a filesystem mounted on the DMF server itself. This can be a local filesystem or a remote one mounted through NFS or similar filesharing protocol. If the disk MSP is configured as a DCM, the filesystem used by the DCM must be a local XFS filesystem.

Most commonly, the offline media is magnetic tape, usually in a tape library (also known as a robotic library or silo). DMF has two tape components: the tape MSP (`dmatmsp`) and the library server (`dmatls`).

| Note: The tape MSP has limitations in some environments and has been superseded by the newer LS.

Figure 1-2 summarizes the various MSP types.

Figure 1-2. MSP Types

Figure 1-3 shows the architecture of the LS and tape MSP.

Figure 1-3. Library Server and Tape MSP Architecture

There is one LS process (`dmatls`) per tape library, which maintains a pair of databases that all of its components share. The entities in the shaded boxes in Figure 1-3 are internal components of the `dmatls` process. Their functions are as follows:

Drive group |  | The drive group is responsible for the management of a group of interchangeable tape drives located in the one tape library. These drives can be used by multiple volume groups (see volume groups below) and by non-LS users, such as MSPs, and non-DMF processes, such as backups and interactive users. However, in the latter cases, the drive group has no management involvement; the mounting service (TMF or OpenVault) is responsible for ensuring that these possibly competing uses of the tape drives do not interfere with each other. The main task of the drive group is to monitor tape I/O for errors, attempt to classify them as volume, drive, or mounting service problems, and to take preventive action.
Volume group |  | The volume group holds at most one copy of user files on a pool of tape volumes, of which it has exclusive use. It can use only the tape drives managed by a single drive group.
Allocation group |  | The allocation group is really a special type of volume group, used to hold a communal pool of empty tapes. These tapes can be transferred to a volume group as they are needed, and can be returned when empty again. Use of an allocation group is optional.
Resource scheduler |  | In a busy environment, it is common for the number of drives requested by volume groups to exceed the number available. The purpose of the resource scheduler is to decide which volume groups should have first access to drives as they become available, and which should wait, and to advise the drive group of the result. The DMF administrator can configure the resource scheduler to meet site requirements.
Resource scheduler algorithm |  | Given the wide variety of site requirements, sites can write their own scheduling routines in C++. These routines are packaged in a dynamically-loadable Dynamic Shared Object library (DSO or `.so` file). When loaded, these routines are an internal component of the `dmatls` process. In the absence of a site-supplied algorithm, standard algorithms are provided with DMF.
Resource watcher |  | The resource watcher monitors the activity of the other components, and frequently updates files that contain data of use to the administrator. The main format is HTML files viewable by a web browser, but text files designed for use by `awk` or `perl` scripts are also maintained.

In contrast to the LS process, each tape MSP has its own database of tape volumes it controls and the user files (at most one copy of each) that they contain. It is somewhat similar to the volume group previously described. Tape MSPs refer to a "device object," which controls a group of tape drives in a similar, but less flexible, way as the drive group previously described. A site can use any combination of the various MSPs or LSs; they are not mutually exclusive. Exception: The tape MSP is not supported on Linux systems.

The `dmatrc` and `dmatwc`. These processes are called the read- and write-children, and are created by MSPs and volume groups to perform the actual reading and writing of tapes. Unlike most of the other DMF processes that run indefinitely, these processes are created as needed, and are terminated when their specific work has been completed.

Media transports and robotic automounters are also key components of all DMF installations. Generally, DMF can be used with any transport and automounter that is supported by either OpenVault or TMF. Additionally, DMF supports absolute block positioning, a media transport capability that allows rapid positioning to an absolute block address on the tape volume. When this capability is provided by the transport, positioning speed is often three times faster than that obtained when reading the volume to the specified position. For details, see “Hardware and Software Requirements”.

## Ensuring Data Integrity

DMF provides capabilities ensure the integrity of offline data. For example, you can have multiple MSPs or volume groups with each managing its own pool of media volumes. Therefore, you can configure DMF to copy filesystem data to multiple offline locations.

DMF stores data that originates in a CXFS or XFS filesystem. (You can also convert other file servers to IRIX or Linux file servers running DMF.) Each object stored corresponds to a file in the native filesystem. When a user deletes a file, the inode for that file is removed from the filesystem. Deleting a file that has been migrated begins the process of invalidating the offline image of that file. In the tape MSP or LS, this eventually creates a gap in the migration medium. To ensure effective use of media, the LS provides a mechanism for reclaiming space lost to invalid data. This process is called volume merging.

Much of the work done by DMF involves transaction processing that is recorded in databases. The DMF database provides for full transaction journaling and employs two-phase commit technology. The combination of these two features ensures that DMF applies only whole transactions to its database. Additionally, in the event of an unscheduled system interrupt, it is always possible to replay the database journals in order to restore consistency between the DMF databases and the filesystem. DMF utilities also allow you to verify the general integrity of the DMF databases themselves.

## DMF Architecture

DMF consists of the DMF daemon and one or more MSPs or LSs. The DMF daemon accepts requests from the DMF administrator or from users to migrate filesystem data, and communicates with the operating system kernel to maintain a file's migration state in that file's inode.

The DMF daemon is responsible for dispensing a unique bit file identifier (BFID) for each file that is migrated. The daemon also determines the destination of migration data and forms requests to the appropriate MSP/LS to make offline copies.

The MSP/LS accepts requests from the DMF daemon. For outbound data, the MSP/LS accrues requests until the volume of data justifies a volume mount. Requests for data retrieval are satisfied as they arrive. When multiple retrieval requests involve the same volume, all file data is retrieved in a single pass across the volume.

DMF uses the kernel interface defined by the Data Management Interface Group (DMIG). DMAPI is also supported by X/Open, where it is evolving as the XDSM standard.

Figure 1-4 illustrates the DMF architecture.

Figure 1-4. DMF Architecture

## Capacity and Overhead

DMF has evolved in production-oriented, customer environments. It is designed to make full use of parallel and asynchronous operations, and to consume minimal system overhead while it executes, even in busy environments in which files are constantly moving online or offline. Exceptions to this rule will occasionally occur during infrequent maintenance operations when a full scan of filesystems or databases is performed.

The capacity of DMF is measured in several ways, as follows:

- Total number of files. The DMF daemon database addressing limits the size of the daemon database to approximately 4 billion entries. There is one database entry for each copy of a file that DMF manages. Therefore, if a site makes two copies of each DMF-managed file, DMF can manage approximately 2 billion files.

- Total volume of data. Capacity in data volume is limited only by the physical environment and the density of media.

- Total volume of data moved between online and offline media. The number of tape drives configured for DMF, the number of tape channels, and the number of disk channels all figure highly in the effective bandwidth. In general, DMF provides full-channel performance to both tape and disk.

- Storage capacity. DMF can support any file that can be created on the CXFS or XFS filesystem being managed.

## DMF Administration

DMF can be configured for a variety of environments including the following:

- Support of batch and interactive processing in a general-purpose environment with limited disk space

- Dedicated file servers

- Lights-out operations

DMF manages two primary resources: pools of offline media and free space on native filesystems.

As a DMF administrator, you must characterize and determine the size of the environment in which DMF will run. You should plan for a certain capacity, both in the number of files and in the volume of data. You should also estimate the rate at which you will be moving data between the DMF store and the native filesystem. You should select autoloaders and media transports that are suitable for the data volume and delivery rates you anticipate.

Beyond initial planning and setup, DMF requires that you perform recurring administrative duties. DMF allows you to configure tasks that automate these duties. A task is a cron-like process initiated on a time schedule you determine. Configuration tasks are defined with configuration file parameters. The tasks are described in detail in “Configuring Daemon Maintenance Tasks” in Chapter 2, and “Configuring Maintenance Tasks for Tape MSP and LS” in Chapter 2.

DMF requires administrative duties to be performed in the following areas:

- File ranking. You must decide which files are most important as migration candidates. When DMF migrates and frees files, it chooses files based on criteria you chose. The ordered list of files is called the DMF candidate list. Whenever DMF responds to a critical space threshold, it builds a new migration candidate list for the filesystem that reached the threshold. See “Generating the Candidate List” in Chapter 3.

- Automated space management. You must decide how much free space to maintain on each managed filesystem. DMF has the ability to monitor filesystem capacity and to initiate file migration and the freeing of space when free space falls below the prescribed thresholds. See Chapter 3, “Automated Space Management”.

- Offline data management. DMF offers the ability to migrate data to multiple offline locations. Each location is managed by a separate MSP or volume group and is usually constrained to a specific type of medium.

  Complex strategies are possible when using multiple MSPs, LSs, or volume groups. For example, short files can be migrated to a device with rapid mount times, while long files can be routed to a device with extremely high density.

  You can describe criteria for MSP or volume group selection. When setting up a tape MSP or volume group, you assign a pool of tapes for use by that MSP/volume group. The `dmvoladm(8)` utility provides management of the tape MSP/LS media pools.

  You can configure DMF to automatically merge tapes that are becoming sparse--that is, full of data that has been deleted by the owner. With this configuration (using the `run_merge_tapes.sh` task), the media pool is merged on a regular basis in order to reclaim unusable space.

  Recording media eventually becomes unreliable. Sometimes, media transports become misaligned so that a volume written on one cannot be read from another. Two utilities are provided that support management of failing media. The `dmatsnf(8)` utility is used to scan a DMF volume for flaws, and `dmatread (8)` is used for recovering data. Additionally, the volume merge process built into the MSP/LS is capable of effectively recovering data from failed media.

  Chapter 6, “Media-Specific Processes and Library Servers”, provides more information on administration.

- Integrity and reliability. Integrity of data is a central concern to the DMF administrator. You must understand and monitor processes in order to achieve the highest levels of data integrity, as follows:

  - Even though you are running DMF, you must still run backups because DMF moves only the data associated with files, not the file inodes or directories. You can configure DMF to automatically run backups of your DMF-managed filesystems.

    The `xfsdump` and `xfsrestore` utilities understand when a file is fully migrated. The `xfsdump` utility has an option that allows for dumping only files that are not fully migrated. Files that are dual-state or offline have only their inodes backed up.

    You can establish a policy of migrating 100% of DMF-managed filesystems, thereby leaving only a small volume of data that the dump utility must record. This practice can greatly increase the availability of the machine on which DMF is running because, generally, dump commands must be executed in a quiet environment.

    You can configure the `run_full_dump.sh` and `run_partial_dump.sh` tasks to ensure that all files have been migrated. These tasks can be configured to run when the environment is quiet.

  - DMF databases record all information about stored data. The DMF databases must be synchronized with the filesystems DMF manages. Much of the work done by DMF ensures that the DMF databases remain aligned with the filesystems.

    You can configure DMF to automatically examine the consistency and integrity of the DMF daemon and MSP/LS databases. You can configure DMF to periodically copy the databases to other devices on the system to protect them from loss (using the `run_copy_databases.sh` task). This task also uses the the `dmdbcheck` utility to ensure the integrity of the databases before saving them.

    DMF uses journal files to record database transactions. Journals can be replayed in the event of an unscheduled system interrupt. You must ensure that journals are retained in a safe place until a full backup of the DMF databases can be performed.

    You can configure the `run_remove_logs.sh` and `run_remove_journals.sh` tasks to automatically remove old logs and journals, which will prevent the DMF `SPOOL_DIR` directory from overflowing.

  You can configure the `run_hard_delete.sh` task to automatically perform hard-deletes, which are described in “Recalling a Migrated File”.

## The User's View of DMF

While the administrator has access to a wide variety of commands for controlling DMF, the end user sees very little. Migrated files remain cataloged in their original directories and are accessed as if they were still on online disk. The only difference users might notice is a delay in access time.

However, commands are provided for file owners to affect the manual storing and retrieval of data. Users can do the following:

- Explicitly migrate files by using the `dmput(1)` command

- Explicitly recall files by using the `dmget(1)` command

- Copy all or part of the data from a migrated file to an online file by using the `dmcopy(1)` command

- Determine whether a file is migrated by using the `dmfind(1)` or `dmls(1)` commands

- Test in shell scripts whether a file is online or offline by using the `dmattr(1)` command

## DMF File Concepts and Terms

DMF regards files as being one of the following:

- Regular files are user files residing only on online disk

- Migrating files are files whose offline copies are in progress

- Migrated files can be either of the following:

  - Dual-state files are files whose data resides both on online disk and on secondary storage

  - Offline files are files whose data is no longer on online disk

  - Unmigrating files are previously offline files in the process of being recalled to online disk

DMF does not migrate pipes, directories, or UNIX special files.

Like a regular file, a migrated file has an inode. Only an offline file requires the intervention of the DMF daemon to access its data; a dual-state file is accessed directly from the online disk copy.

The operating system informs the DMF daemon when a migrated file is modified. If anything is written to a migrated file, the offline copy is no longer valid, and the file becomes a regular file until it is migrated again.

### Migrating a File

A file is migrated when the automated space management controller `dmfsmon(8)` selects the file or when an owner requests that the file be migrated by using the `dmput(1)` command.

The DMF daemon keeps a record of all migrated files in its database. The key to each file is its bit file identifier (BFID). For each migrated file, the daemon assigns a BFID that is stored in the file's inode.

When the daemon receives a request to migrate a file, it adjusts the state of the file, ensures that the necessary MSPs or volume groups are active, and sends a request to the MSPs or volume groups. MSPs or volume groups then copy data to the offline storage media.

When the MSPs or volume groups have completed the offline copies, the daemon marks the file as fully migrated in its database and changes the file to dual-state. If the user specified the `dmput -r` option, or if `dmfsmon` requested that the file's space be released, the daemon releases the data blocks and changes the user file state to offline.

### Recalling a Migrated File

When a migrated file must be recalled, a request is made to the DMF daemon. The daemon selects an MSP or volume group from its internal list and sends that MSP/volume group a request to recall a copy of the file. If more than one MSP or volume group has a copy, the first one in the list is used. (The list is created from the configuration file.)

After a user has modified or removed a migrated file, its bit file identifier (BFID) is soft-deleted. A file is soft-deleted when it is logically deleted from the daemon database. This is accomplished by setting the delete date field in the database to the current date and time for each entry referring to the modified or removed file.

A file is hard-deleted when its BFID is physically removed from the DMF database. You can configure DMF to automatically perform hard-deletes. This is done using the `run_hard_delete.sh` task, which uses the `dmhdelete(8)` utility.

The soft-delete state allows for the possibility that the filesystem might be restored after the user has removed a file. When a filesystem is reloaded from a dump image, it is restored to a state at an earlier point in time. A file that had been migrated and then removed might become migrated again due to the restore operation. This can create serious problems if the database entries for the file have been physically deleted (hard-deleted). In this case, the user would receive an error when trying to open the file because the file cannot be retrieved.

Do not hard-delete a database entry until after you are sure that the corresponding files will never be restored. Hard-delete requests are sent to the relevant MSPs and volume groups so that copies of the file can be removed from media. For a tape MSP/volume group, this involves compression (or merging).

## Command Overview

The following section provides definitions for administrator commands grouped by function.

### Configuration Commands

The configuration file, `/etc/dmf/dmf.conf` , contains configuration objects and associated configuration parameters that control the way DMF operates. By changing the values associated with these objects and parameters, you can modify the behavior of DMF.

For information about editing the configuration file, see Chapter 2, “Configuring DMF”. The following man pages are related to the configuration file:

Man page | Description
`dmf.conf(5)` | Describes the DMF configuration objects and parameters in detail
`dmconfig(8)` | Prints DMF configuration parameters to standard output

### DMF Daemon and Related Commands

The DMF daemon, `dmfdaemon(8)`, communicates with the kernel through a device driver and receives backup and recall requests from users through a socket. The daemon activates the appropriate MSPs and LSs for file migration and recall, maintaining communication with them through unnamed pipes. It also changes the state of inodes as they pass through each phase of the migration and recall process. In addition, `dmfdaemon` maintains a database containing entries for every migrated file on the system. Updates to database entries are logged in a journal file for recovery. See Chapter 4, “The DMF Daemon”, for a detailed description of the DMF daemon.

| Caution: If used improperly, commands that make changes to the DMF database can cause data to be lost.

The following administrator commands are related to `dmfdaemon` and the daemon database:

Command |  | Description
`dmaudit(8)` |  | Reports discrepancies between filesystems and the daemon database. This command is executed automatically if you configure the `run_audit.sh` task.
`dmcheck(8)` |  | Checks the DMF installation and configuration and reports any problems.
`dmdadm(8)` |  | Performs daemon database administrative functions, such as viewing individual database records.
`dmfdaemon(8)` |  | Starts the DMF daemon. The preferred method is to use the `/etc/init.d/dmf` script.
`dmdbcheck(8)` |  | Checks the consistency of a database by validating the location and key values associated with each record and key in the data and key files (also an MSP/LS command). If you configure the `run_copy_database.sh` task, this command is executed automatically as part of the task. The consistency check is completed before the DMF databases are saved.
`dmdbrecover(8)` |  | Updates the daemon and tape MSP/LS databases with journal entries.
`dmdidle(8)` |  | Causes files not yet copied to tape to be flushed to tape, even if this means forcing only a small amount of data to a volume.
`dmdstat(8)` |  | Indicates to the caller the current status of `dmfdaemon`.
`dmdstop(8)` |  | Causes `dmfdaemon` to shut down.
`dmhdelete(8)` |  | Deletes expired daemon database entries and releases corresponding MSP or volume group space, resulting in logically less active data. This command is executed automatically if you configure the `run_hard_delete.sh` task.
`dmmigrate(8)` |  | Migrates regular files that match specified criteria in the specified filesystems, leaving them as dual-state. This utility is often used to migrate files before running backups of a filesystem, hence minimizing the size of the dump image. It may also be used in a DCM environment to force cache files to be copied to tape if necessary.
`dmsnap(8)` |  | Copies the DMF daemon and the MSP/LS databases to a specified location. If you configure the `run_copy_database.sh` task, this command is executed automatically as part of the task.
`dmversion(8)` |  | Reports the version of DMF that is currently executing.

### Space Management Commands

The following commands are associated with automated space management, which allows DMF to maintain a specified level of free space on a filesystem through automatic file migration:

Command | Description
`dmfsfree(8)` | Attempts to bring the free space and migrated space of a filesystem into compliance with configured values.
`dmfsmon(8)` | Monitors the free space levels in filesystems configured with automated space management is enabled (as `auto`) and lets you maintain a specified level of free space.
`dmscanfs(8)` | Scans DMF filesystems or DCM caches and prints status information to `stdout`.

See Chapter 3, “Automated Space Management”, for details.

### MSP/LS Commands

The DMF tape MSP and LS maintain a database that contains volume (VOL) records and catalog (CAT) records. VOL records contain information about tape volumes, and CAT records contain information about offline copies of migrated files.

The disk and FTP MSPs allow the use of local or remote disk storage for storing migrated data. They use no special commands, utilities, or databases. For more information, see “Disk MSP” in Chapter 6, and “FTP MSP” in Chapter 6.

The following commands manage the CAT and VOL records for the tape MSP/LS:

Command | Description
`dmcatadm(8)` | Provides maintenance and recovery services for the CAT database.
`dmvoladm(8)` | Provides maintenance and recovery services for the VOL database, including the selection of volumes for tape merge operations.

Most data transfers to and from tape media are performed by components internal to the MSP/LS. However, there are also two utilities that can read tape MSP/LS volumes directly:

Command | Description
`dmatread(8)` | Copies data directly from MSP/LS volumes to disk.
`dmatsnf(8)` | Audits and verifies the format of MSP/LS volumes.

There are also tools that check for MSP database inconsistencies:

Command | Description
`dmatvfy(8)` | Verifies the MSP/LS database contents against the `dmfdaemon(8)` database. This command is executed automatically if you configure the `run_audit.sh` task.
`dmdbcheck(8)` | Checks the consistency of a database by validating the location and key values associated with each record and key in the data and key files.

### Disk Cache Manager (DCM) Commands

The following commands support the DCM:

Command | Description
`dmdskfree(8)` | Manages file space within the disk cache and as needed migrates files to tape or removes them from the disk cache.

### Commands for Other Utilities

The following utilities are also available:

Command | Description
`dmclripc(8)` | Frees system interprocess communication (IPC) resources and token files used by `dmlockmgr` and its clients when abnormal termination prevents orderly exit processing.
`dmcollect(8)` | Collects relevant details for problem analysis when DMF is not functioning properly. You should run this command before submitting a bug report to DMF support, should this ever be necessary.
`dmdate(8)` | Performs calculations on dates for administrative support scripts.
`dmdump(8)` | Creates a text copy of an inactive database file or a text copy of an inactive complete DMF daemon database.
`dmdumpj(8)` | Creates a text copy of DMF journal transactions.
`dmfill(8)` | Recalls migrated files to fill a percentage of a filesystem. This command is mainly used in conjunction with dump and restore commands to return a corrupted filesystem to a previously known valid state.
`dmlockmgr(8)` | Invokes the database lock manager. The lock manager is an independent process that communicates with all applications that use the DMF database, mediates record lock requests, and facilitates the automatic transaction recovery mechanism.
`dmmove(8)` | Moves copies of a migrated file's data to the specified MSPs/volume groups.
`dmmaint(8)` | Performs DMF maintenance and provides interfaces for licensing and initial configuration.
`dmov_keyfile(8)` | Creates the file of DMF OpenVault keys, ensuring that the contents of the file are semantically correct and have the correct file permissions. This command removes any DMF keys in the file for the OpenVault server system and adds new keys at the front of the file.
`dmov_loadtapes(8)` | Scans a tape library for volumes not imported into the OpenVault database and allows the user to select a portion of them to be used by an MSP/volume group. The selected tapes are imported into the OpenVault database, assigned to the DMF application, and added to the MSP's/LS's database.
`dmov_makecarts(8)` | Makes the tapes in one or more MSP/LS databases accessible through OpenVault by importing into the OpenVault database any tapes unknown to it and by registering all volumes to the DMF application not yet so assigned.
`dmselect(8)` | Selects migrated files based on given criteria. The output of this command can be used as input to `dmmove (8)`.
`dmsort(8)` | Sorts files of blocked records.
`dmxfsrestore(8)` | Calls the `xfsrestore(1M)` command to restore files dumped to tape volumes that were produced by DMF administrative maintenance scripts.

Prev | Table of Contents | Next
About This Guide |  | Chapter 2. Configuring DMF