This chapter provides an overview of the Data Migration Facility (DMF) and its administration. It discusses the following:
DMF is a hierarchical storage management system for SGI environments. DMF allows you to oversubscribe your online disk in a manner that is transparent to users; a user cannot determine, by using POSIX-compliant commands for filesystem enquiry, whether a file is online or offline. Only when special commands or command options are used can a file's actual residence be determined. This transparent migration is possible because DMF leaves inodes and directories intact within the native filesystem.
DMF automatically detects a drop below the filesystem free-space threshold and migrates selected data from expensive online disk to cheaper secondary storage, such as tapes. DMF automatically recalls the file data from offline media when the user accesses the file with normal operating system commands. You can also manually force a file to be migrated or recalled.
DMF can migrate data to the following:
Figure 1-1 provides a conceptual overview of the data flow between applications and storage media.
DMF supports a range of storage management applications. In some environments, DMF is used strictly to manage highly stressed online disk resources. In other environments, it is also used as an organizational tool for safely managing large volumes of offline data. In all environments, DMF scales to the storage application and to the characteristics of the available storage devices.
DMF interoperates with the following:
By combining these services with DMF, you can configure an SGI system as a high-performance fileserver.
DMF transports large volumes of data on behalf of many users. Because system interrupts and occasional storage device failures cannot be avoided, it is essential that the safety and integrity of data be verifiable. Therefore, DMF also provides tools necessary to validate your storage environment.
DMF has evolved around these customer requirements for scalability and the safety of data. As a filesystem migrator, DMF manages the capacity of online disk resources by transparently moving file data from disk to offline media. Most commonly, the secondary storage is tape, managed by OpenVault or the Tape Management Facility (TMF). However, the secondary storage can be any bulk-storage device accessible locally through NFS or FTP.
DMF includes the following software subsystems:
Server, which provides the full set of DMF functionality, including the DMF daemon, infrastructure, user and administrator commands, online manuals, and all man pages. This applies to SGI ProPack for Linux systems on SGI Altix servers and to IRIX systems. You should install this subsystem on only those IRIX and SGI ProPack machines from which you will administer DMF, including NFS servers and potential CXFS metadata servers.
Client, which provides the limited set of user commands, libraries, and a subset of the man pages. This applies to all supported operating systems (see “Hardware and Software Requirements”). You should install this subsystem on machines from which you want to give users access to DMF user commands, such as dmput and dmget. IRIX and SGI ProPack machines that are CXFS client-only nodes or NFS clients should use this subsystem; for other supported operating systems, such as Solaris, the client subsystem is the only option.
Only one of these subsystems can be installed on a given machine.
The DMF server subsystem runs on the following operating systems:
 | Note: For the currently supported release levels, libraries, and tape devices, see the DMF.Readme file. |
The DMF client subsystem supports nodes running the following operating systems:
The client-only user commands are as follows:
| dmattr |
| dmcopy |
| dmdu |
| dmget |
| dmfind |
| dmls |
| dmput |
| dmtag |
The DMF libdmfusr.so user library lets you write your own site-defined DMF user commands that use the same application program interface (API) as the above DMF user commands.
For filesystems to be managed by DMF, they must be mounted on the DMF server in order to enable the Data Management API (DMAPI) interface. Do one of the following for each platform:
IRIX:
Linux:
Use the following command:
mount -o dmi -o mtpt = mountpoint
Add dmi, mtpt = mountpoint to the fourth field in the fstab entry
For more information, see the mount and fstab man pages.
You must have a separate license for each DMF server (no licensing is required on the client host):
For IRIX and SGI ProPack 4 systems, DMF servers use software licensing based on the FLEXlm product from Macrovision Corporation.
For SGI ProPack 5 systems, DMF servers use software licensing based on License Keys (LK). LK is an SGI Linux software licensing product.
Software keys are used to enforce licensing. DMF licenses apply to a single specific system. DMF license fees vary depending on the amount of data being managed.
When you order DMF, you will receive an entitlement ID. When requesting your permanent DMF license, you must submit the required information for your system type (this information may include information such as the unique system identifier, system serial number, hostname, and entitlement ID). To determine the required information, launch dmmaint and select License Info.
To obtain your permanent DMF license, follow the instructions on the key generation page. After the required information is provided, a key will be generated and displayed on the webpage along with installation instructions. You can use the Update License button on the dmmaint GUI to install the license.
For more information about licensing, see the following webpage:
As a DMF administrator, you determine how disk space capacity is handled by selecting the filesystems that DMF will manage and by specifying the volume of free space that will be maintained on each filesystem. Space management begins with a list of user files that are ranked according to criteria you define. File size and file age are among the most common ranking criteria.
File migration occurs in two stages:
Stage One: A file is copied (migrated) to secondary storage.
Stage Two: After the copy is secure, the file is eligible to have its data blocks released (this usually occurs only after a minimum space threshold is reached).
A file with all offline copies completed is called fully migrated. A file that is fully migrated but whose data blocks have not yet been released is called a dual-state file; its data exists both online and offline, simultaneously. After a file's data blocks have been released, the file is called an offline file.
You choose both the percentage of filesystem volume to migrate and the volume of free space. You can trigger file migration, or file owners can issue manual migration requests.
Offline media is the destination of all migrated data and is managed by daemon-like DMF components called the library server (LS) and the media-specific process (MSP):
LS (dmatls) transfers to and from magnetic tape in a tape library (also known as a robotic library or silo).
FTP MSP (dmftpmsp ) uses the file transfer protocol to transfer to and from disks of another system on the network.
Disk MSP (dmdskmsp ) uses a filesystem mounted on the DMF server itself. This can be a local filesystem or a remote one mounted through NFS or similar filesharing protocol.
Disk cache manager (DCM) is the disk MSP configured for n-tier capability. DMF can manage the disk MSP's storage filesystem and further migrate it to tape, thereby using a slower and less-expensive dedicated filesystem as a cache to improve the performance when recalling files. If the disk MSP is configured as a DCM, the filesystem used by the DCM must be a local XFS filesystem.
A site can use any combination of LS, disk MSP, FTP MSP, or DCM; they are not mutually exclusive.
Figure 1-2 summarizes using DMF.
Figure 1-3 shows the architecture of the LS.
There is one LS process (dmatls) per tape library, which maintains a database that all of its components share. The entities in the shaded boxes in Figure 1-3 are internal components of the dmatls process. Their functions are as follows:
The dmatrc and dmatwc processes are called the read- and write-children. They are created by volume groups to perform the actual reading and writing of tapes. Unlike most of the other DMF processes that run indefinitely, these processes are created as needed, and are terminated when their specific work has been completed.
Media transports and robotic automounters are also key components of all DMF installations. Generally, DMF can be used with any transport and automounter that is supported by either OpenVault or TMF. Additionally, DMF supports absolute block positioning, a media transport capability that allows rapid positioning to an absolute block address on the tape volume. When this capability is provided by the transport, positioning speed is often three times faster than that obtained when reading the volume to the specified position. For details, see “Hardware and Software Requirements”.
DMF provides capabilities to ensure the integrity of offline data. For example, you can have multiple MSPs or volume groups with each managing its own pool of media volumes. Therefore, you can configure DMF to copy filesystem data to multiple offline locations.
DMF stores data that originates in a CXFS or XFS filesystem. (You can also convert other fileservers to IRIX or Linux fileservers running DMF.) Each object stored corresponds to a file in the native filesystem. When a user deletes a file, the inode for that file is removed from the filesystem. Deleting a file that has been migrated begins the process of invalidating the offline image of that file. In the LS, this eventually creates a gap in the migration medium. To ensure effective use of media, the LS provides a mechanism for reclaiming space lost to invalid data. This process is called volume merging.
Much of the work done by DMF involves transaction processing that is recorded in databases. The DMF database provides for full transaction journaling and employs two-phase commit technology. The combination of these two features ensures that DMF applies only whole transactions to its database. Additionally, in the event of an unscheduled system interrupt, it is always possible to replay the database journals in order to restore consistency between the DMF databases and the filesystem. DMF utilities also allow you to verify the general integrity of the DMF databases themselves.
See “DMF Administration” for more information
DMF consists of the DMF daemon and one or more MSPs or LSs. The DMF daemon accepts requests from the DMF administrator or from users to migrate filesystem data, and communicates with the operating system kernel to maintain a file's migration state in that file's inode.
The DMF daemon is responsible for dispensing a unique bit file identifier (BFID) for each file that is migrated. The daemon also determines the destination of migration data and forms requests to the appropriate MSP/LS to make offline copies.
The MSP/LS accepts requests from the DMF daemon. For outbound data, the LS accrues requests until the volume of data justifies a volume mount. Requests for data retrieval are satisfied as they arrive. When multiple retrieval requests involve the same volume, all file data is retrieved in a single pass across the volume.
DMF uses the DMAPI kernel interface defined by the Data Management Interface Group (DMIG). DMAPI is also supported by X/Open, where it is known as the XDSM standard.
Figure 1-4 illustrates the DMF architecture.
DMF has evolved in production-oriented, customer environments. It is designed to make full use of parallel and asynchronous operations, and to consume minimal system overhead while it executes, even in busy environments in which files are constantly moving online or offline. Exceptions to this rule will occasionally occur during infrequent maintenance operations when a full scan of filesystems or databases is performed.
The capacity of DMF is measured in several ways, as follows:
Total number of files. The DMF daemon database addressing limits the size of the daemon database to approximately 4 billion entries. There is one database entry for each copy of a file that DMF manages. Therefore, if a site makes two copies of each DMF-managed file, DMF can manage approximately 2 billion files.
Total volume of data. Capacity in data volume is limited only by the physical environment and the density of media.
Total volume of data moved between online and offline media. The number of tape drives configured for DMF, the number of tape channels, and the number of disk channels all figure highly in the effective bandwidth. In general, DMF provides full-channel performance to both tape and disk.
Storage capacity. DMF can support any file that can be created on the CXFS or XFS filesystem being managed.
DMF can be configured for a variety of environments including the following:
DMF manages two primary resources: pools of offline media and free space on native filesystems.
As a DMF administrator, you must characterize and determine the size of the environment in which DMF will run. You should plan for a certain capacity, both in the number of files and in the volume of data. You should also estimate the rate at which you will be moving data between the DMF store and the native filesystem. You should select autoloaders and media transports that are suitable for the data volume and delivery rates you anticipate.
Beyond initial planning and setup, DMF requires that you perform recurring administrative duties. DMF allows you to configure tasks that automate these duties. A task is a cron-like process initiated on a time schedule you determine. Configuration tasks are defined with configuration file parameters. The tasks are described in detail in “Configuring Daemon Maintenance Tasks” in Chapter 2 and “Configuring Maintenance Tasks for the LS” in Chapter 2.
DMF requires administrative duties to be performed in the following areas:
File ranking. You must decide which files are most important as migration candidates. When DMF migrates and frees files, it chooses files based on criteria you chose. The ordered list of files is called the DMF candidate list. Whenever DMF responds to a critical space threshold, it builds a new migration candidate list for the filesystem that reached the threshold. See “Generating the Candidate List” in Chapter 3.
Automated space management. You must decide how much free space to maintain on each managed filesystem. DMF has the ability to monitor filesystem capacity and to initiate file migration and the freeing of space when free space falls below the prescribed thresholds. See Chapter 3, “Automated Space Management”.
Offline data management. DMF offers the ability to migrate data to multiple offline locations. Each location is managed by a separate MSP or volume group and is usually constrained to a specific type of medium.
Complex strategies are possible when using multiple MSPs, LSs, or volume groups. For example, short files can be migrated to a device with rapid mount times, while long files can be routed to a device with extremely high density.
You can describe criteria for MSP or volume group selection. When setting up a volume group, you assign a pool of tapes for use by that volume group. The dmvoladm(8) utility provides management of the LS media pools.
You can configure DMF to automatically merge tapes that are becoming sparse--that is, full of data that has been deleted by the owner. With this configuration (using the run_merge_tapes.sh task), the media pool is merged on a regular basis in order to reclaim unusable space.
Recording media eventually becomes unreliable. Sometimes, media transports become misaligned so that a volume written on one cannot be read from another. Two utilities are provided that support management of failing media:
dmatsnf(8) utility is used to scan a DMF volume for flaws
dmatread(8) is used for recovering data.
Additionally, the volume merge process built into the LS is capable of effectively recovering data from failed media.
Chapter 6, “Media-Specific Processes and Library Servers”, provides more information on administration.
Integrity and reliability. Integrity of data is a central concern to the DMF administrator. You must understand and monitor processes in order to achieve the highest levels of data integrity, as follows:
Even though you are running DMF, you must still run backups because DMF moves only the data associated with files, not the file inodes or directories. You can configure DMF to automatically run backups of your DMF-managed filesystems.
The xfsdump and xfsrestore utilities understand when a file is fully migrated. The xfsdump utility has an option that allows for dumping only files that are not fully migrated. Files that are dual-state or offline have only their inodes backed up.
You can establish a policy of migrating 100% of DMF-managed filesystems, thereby leaving only a small volume of data that the dump utility must record. This practice can greatly increase the availability of the machine on which DMF is running because, generally, dump commands must be executed in a quiet environment.
You can configure the run_full_dump.sh and run_partial_dump.sh tasks to ensure that all files have been migrated. These tasks can be configured to run when the environment is quiet.
DMF databases record all information about stored data. The DMF databases must be synchronized with the filesystems DMF manages. Much of the work done by DMF ensures that the DMF databases remain aligned with the filesystems.
You can configure DMF to automatically examine the consistency and integrity of the DMF daemon and LS databases. You can configure DMF to periodically copy the databases to other devices on the system to protect them from loss (using the run_copy_databases.sh task). This task also uses the the dmdbcheck utility to ensure the integrity of the databases before saving them.
DMF uses journal files to record database transactions. Journals can be replayed in the event of an unscheduled system interrupt. You must ensure that journals are retained in a safe place until a full backup of the DMF databases can be performed.
You can configure the run_remove_logs.sh and run_remove_journals.sh tasks to automatically remove old logs and journals, which will prevent the DMF SPOOL_DIR directory from overflowing.
You can configure the run_hard_delete.sh task to automatically perform hard-deletes, which are described in “Recalling a Migrated File”.
While the administrator has access to a wide variety of commands for controlling DMF, the end user sees very little. Migrated files remain cataloged in their original directories and are accessed as if they were still on online disk. The only difference users might notice is a delay in access time.
However, commands are provided for file owners to affect the manual storing and retrieval of data. Users can do the following:
Copy all or part of the data from a migrated file to an online file by using the dmcopy(1) command.
Determine whether a file is migrated by using the dmfind(1) or dmls(1) commands.
Test in shell scripts whether a file is online or offline by using the dmattr(1) command.
Determine the number of blocks contained in specified files and directories on a DMF-managed filesystem by using the dmdu(1) command.
Allow a site-assigned 32-bit integer to be associated with a specific file (which can be tested in the when clause of particular configuration parameters and in site-defined policies) by using the dmtag(1) command. See “Customizing DMF”.
| Note: The functionality of some of these commands can be modified by site-defined policies; see “Customizing DMF”. |
DMF regards files as being one of the following:
Migrating files are files whose offline copies are in progress
Migrated files can be one of the following:
Dual-state files are files whose data resides both on online disk and on secondary storage
Offline files are files whose data is no longer on online disk
Unmigrating files are previously offline files in the process of being recalled to online disk
Partial-state files are files with some combination of dual-state, offline, and/or unmigrating regions (see “Partial-State Files”)
DMF does not migrate pipes, directories, or UNIX special files.
Like a regular file, a migrated file has an inode. An offline file or a partial-state file requires the intervention of the DMF daemon to access its offline data; a dual-state file is accessed directly from the online disk copy.
The operating system informs the DMF daemon when a migrated file is modified. If anything is written to a migrated file, the offline copy is no longer valid, and the file becomes a regular file until it is migrated again.
A file is migrated when the automated space management controller dmfsmon(8) selects the file or when an owner requests that the file be migrated by using the dmput(1) command.
The DMF daemon keeps a record of all migrated files in its database. The key to each file is its bit file identifier (BFID). For each migrated file, the daemon assigns a BFID that is stored in the file's inode.
When the daemon receives a request to migrate a file, it adjusts the state of the file, ensures that the necessary MSPs or volume groups are active, and sends a request to the MSPs or volume groups. MSPs or volume groups then copy data to the offline storage media.
When the MSPs or volume groups have completed the offline copies, the daemon marks the file as fully migrated in its database and changes the file to dual-state. If the user specified the dmput -r option, or if dmfsmon requested that the file's space be released, the daemon releases the data blocks and changes the user file state to offline.
For more information, see the dmput man page.
When a migrated file must be recalled, a request is made to the DMF daemon. The daemon selects an MSP or volume group from its internal list and sends that MSP/volume group a request to recall a copy of the file. If more than one MSP or volume group has a copy, the first one in the list is used. (The list is created from the configuration file.)
After a user has modified or removed a migrated file, its bit file identifier (BFID) is soft-deleted, meaning that it is logically deleted from the daemon database. This is accomplished by setting the delete date field in the database to the current date and time for each entry referring to the modified or removed file.
A file is hard-deleted when its BFID is physically removed from the DMF database. You can configure DMF to automatically perform hard-deletes. This is done using the run_hard_delete.sh task, which uses the dmhdelete(8) utility.
The soft-delete state allows for the possibility that the filesystem might be restored after the user has removed a file. When a filesystem is reloaded from a dump image, it is restored to a state at an earlier point in time. A file that had been migrated and then removed might become migrated again due to the restore operation. This can create serious problems if the database entries for the file have been physically deleted (hard-deleted). In this case, the user would receive an error when trying to open the file because the file cannot be retrieved.
Do not hard-delete a database entry until after you are sure that the corresponding files will never be restored. Hard-delete requests are sent to the relevant MSPs and volume groups so that copies of the file can be removed from media. For a volume group, this involves compression (or merging).
The following section provides definitions for administrator commands grouped by function.
| Note: The functionality of some of these commands can be affected by site-defined policies; see “Customizing DMF”. |
The configuration file, /etc/dmf/dmf.conf , contains configuration objects and associated configuration parameters that control the way DMF operates. By changing the values associated with these objects and parameters, you can modify the behavior of DMF.
For information about editing the configuration file, see Chapter 2, “Configuring DMF”. The following man pages are related to the configuration file:
The DMF daemon, dmfdaemon (8), communicates with the kernel through a device driver and receives backup and recall requests from users through a socket. The daemon activates the appropriate MSPs and LSs for file migration and recall, maintaining communication with them through unnamed pipes. It also changes the state of inodes as they pass through each phase of the migration and recall process. In addition, dmfdaemon maintains a database containing entries for every migrated file on the system. Updates to database entries are logged in a journal file for recovery. See Chapter 4, “The DMF Daemon”, for a detailed description of the DMF daemon.
 | Caution: If used improperly, commands that make changes to the DMF database can cause data to be lost. |
The following administrator commands are related to dmfdaemon and the daemon database:
The following commands are associated with automated space management, which allows DMF to maintain a specified level of free space on a filesystem through automatic file migration:
See Chapter 3, “Automated Space Management”, for details.
The LS maintains a database that contains the following:
The disk and FTP MSPs allow the use of local or remote disk storage for storing migrated data. They use no special commands, utilities, or databases. For more information, see “Disk MSP” in Chapter 6 and “FTP MSP” in Chapter 6.
The following commands manage the CAT and VOL records for the LS:
Most data transfers to and from tape media are performed by components internal to the LS. However, there are also two utilities that can read LS volumes directly:
| Command | Description |
| dmatread(8) | |
| dmatsnf(8) |
There are also tools that check for LS database inconsistencies:
The following commands support the DCM:
The following utilities are also available:
You can modify the default behavior of DMF as follows:
File tagging allows an arbitrary 32-bit integer to be associated with specific files so that they can be subsequently identified and acted upon. The specific values are chosen by the site; they have no meaning to DMF.
Non-root users may only set or change a tag value on files that they own, but the root may do this on any files. The files may or may not have been previously migrated.
To set a tag, use the dmtag(1) command or the libdmfusr.so library. For example:
% dmtag -t 42 myfile
To view the tag set for a given file, use the dmtag or dmattr commands. For example:
% dmtag myfile 42 myfile % dmattr -a sitetag myfile 42
A file's tag (if any) can be tested in the when clause of the following configuration parameters by using the keyword sitetag:
AGE_WEIGHT CACHE_AGE_WEIGHT CACHE_SPACE_WEIGHT SELECT_LOWER_VG SELECT_MSP SELECT_VG SPACE_WEIGHT For example:
SELECT_VG fasttape when sitetag = 42
It may also be accessed in site-defined policies, as described below.
For more information, see the dmtag(1) man page and the /CDROM/platform/DMF.Readme file.
Site-defined policies allow you to do site-specific modifications by writing your own library of C++ functions that DMF will consult when making decisions about its operation. For example, you could write a policy that decides at migration time which volume group or MSP an individual file should be sent to, using selection criteria that are specific to your site.
Note: If you customize DMF, you should inform your users so that they can predict how the user commands will work with your policies in place. You can add error, warning, and informational messages for commands so that the user will understand why the behavior of the command differs from the default. For information about the aspects of DMF that may be modified, see the /usr/share/doc/dmf-*/info/sample/sitelib.readme file.
DMF-managed files can have different residency states (online or offline) for different regions of a file. A region is a contiguous range of bytes that have the same residency state. This means that a file can have one region that is online for immediate access and another region that is offline and must be recalled to online media in order to be accessed.
DMF allows for multiple distinct file regions. A file that has more than one region is called a partial-state file. A file that is in a static state (that is, not currently being migrated or unmigrated) can have multiple online and offline regions. You can use the MAX_MANAGED_REGIONS parameter to configure the maximum number of file regions that DMF will allow on a file. You can set this parameter on a per-filesystem basis.
| Note: You should use MAX_MANAGED_REGIONS cautiously. If set capriciously, filesystem scan times can increase greatly. For details about using MAX_MANAGED_REGIONS, see “Configuring Filesystems” in Chapter 2. |
Partial-state files provide the following capabilities:
Accelerated access to first byte, which allows you to access the beginning of an offline file before the entire file has been recalled.
Partial-state file online retention, which allows you to keep a specific region of a file online while freeing the rest of it (for example, if you wanted to keep just the beginning of a file online).
Partial-state file recall, which allows you to recall a specific region of a file without recalling the entire file. For more information, see the dmput(1) and dmget(1) man pages.
You turn off the the partial-state file feature by setting the PARTIAL_STATE_FILES daemon configuration parameter to off. For details, see the information in the DMF.News file.