The Data Migration Facility (DMF) is a comprehensive data management tool that can be used to manage free space on your native XFS file system. DMF can also be used to manage reliable, long-term storage of important data.
DMF accomplishes its work by moving user file data between primary storage and secondary storage. This process is called file migration. File migration can occur transparently to the end user (automatic migration), or it can be invoked manually by command request (manual migration). For a complete description of the capabilities of DMF, its commands, and the role of the IRIX system administrator, please refer to the DMF Administrator's Guide for IRIX.
This chapter describes changes in the DMF 2.6.3 release as well as changes for the 2.6.2.3 update release.
DMF can be used with any SGI platform that runs IRIX versions 6.4.1, 6.5, or later. Both 32-bit and 64-bit architectures are supported. DMF for IRIX systems can manage only XFS file systems.
The following sections describe data storage and management capabilities provided with DMF, as well as the product's interface with the IRIX kernel.
DMF is delivered with a complete set of utilities that can be used to manage the DMF databases and control automatic space management. The DMF component that manages offline data storage is called a media-specific process (or MSP). DMF is delivered with the following MSPs:
A tape MSP that allows the administrator to designate a pool of tapes for use as a repository for stored data
An FTP MSP that allows the administrator to move migrated data to or from any destination machine accessible via the file transfer protocol (FTP)
A disk MSP that migrates data into a directory accessed on the current system. It uses POSIX file interfaces to open, read, write, and close files
Refer to the DMF Administrator's Guide for IRIX for a detailed description of the MSPs.
Several DMF utilities are designed to ensure the integrity of DMF databases and of the media used by the tape MSP. For example, two utilities provided with the tape MSP can be used to scan tape media looking for hard errors and to recover data from failed media. Additionally, the integrity of DMF databases is examined with the following utilities:
dmdbcheck(8), which is used to validate the internal structure of any DMF database
dmaudit(8), which is used to ensure that the DMF database is consistent with the XFS file systems being managed by DMF
You can automate the running of these commands through the DMF configuration file, as described in DMF Administrator's Guide for IRIX. Refer to the DMF Recovery and Troubleshooting Guide for IRIX Systems for a detailed description of the use of dmaudit.
In order for the tape MSP to manage stored data, it must mount and dismount tapes on tape transports connected to the platform on which DMF is running. Most often tape mounting is accomplished by a robotic autoloader. These robotic autoloaders can be managed by OpenVault or by the Tape Management Facility (TMF). DMF supports all the robotic autoloaders the mounting service supports. For a list of supported devices and libraries, refer to the Readmefile accessed by the Dependencies button on the DMF installation interface (dmmaint(8)).
DMF interacts with the IRIX kernel in order to migrate user data. This interaction is accomplished through an interface called the Data Management Application Programming Interface or DMAPI. This interface is recognized by the X/Open Group, where it is called the XDSM standard. DMF is also supported in minor ways by the Network File System (NFS) and Bulk Data Services (BDS). For this reason, you must install several patches on any machine on which you want to run DMF. See the Dependencies button on the DMF installation interface (dmmaint(8)) for additional information about specific patches.
 | Note: For the most current and detailed information on changes in DMF functionality, including bugs fixed with specific releases, refer to the files accessed by the Dependencies and News buttons on the DMF installation interface (dmmaint(8)). |
The following is a summary of key changes that are included in DMF release 2.6.3:
The new dmdbupgrade support script extends the limit on the number of data records in any DMF database (daemon or MSP) from 16.7 million to over 4 billion. The use of the command is described in the file accessed by the News button on the DMF installation interface (dmmaint(8)). You must read the procedures carefully and contact your support representative with any questions.
The MSP write child (dmatwc) has been enhanced in release 2.6.3 to use a separate thread for performing tape I/O. This means that dwatwc can now overlap all of its tape block processing time with its tape write requests, which results in increased performance.
With release 2.6.2.3, the tape MSP supports the use of Sony SY-2120 tape drives and Sony PetaSite/DMS-B35 SCSI-attached robotic libraries over OpenVault. The tape MSP also supports AMPEX DST/DIS series tape drive in combination with the Ampex 712 or other SCSI-attached robotic libraries over OpenVault.
In release 2.6.2.3, the tape MSP was modified to make it more aware of the availability of the mounting service it uses. The MSP checks mounting service availability when it is started and after each occurrence where an MSP write child or read child was unable to reserve its drive. If the mounting service is found to be unavailable, the tape MSP will not start any new child processes until the mounting service is once again available.
If the unavailable mounting service is OpenVault, the MSP will send an email message to the administrator asking that OpenVault be started, and will then periodically poll OpenVault until it becomes available, at which time child processes will again be allowed to run.
In the case of TMF, the tape MSP will not only attempt to initiate the tmdaemon if it is not up (based on the exit status of tmstat), but it will wait until a TMF device in the configuration pending state is configured up before it resumes processing. If TMF cannot be started or if no devices are configured up, the tape MSP will send email to the administrator and poll TMF until a drive becomes available.
Also in release 2.6.2.3, DMF was enhanced to migrate files and free space in a file system that is 100% full. Previously DMF was unable to migrate files when fewer than (approximately) 50 blocks remained in the filesystem. The new functionality depends upon XFS software introduced as part of IRIX release 6.5.2. Prior releases of IRIX do not support this functionality. DMF dynamically checks the IRIX release level and uses the new feature if it is present.
New and changed commands and scripts in DMF release 2.6.2.3 and release 2.6.3 are as follows:
One new command was added in release 2.6.2.3: The dmxfsrestore(8) command lets the DMF administrator invoke the xfsrestore(1m) command to restore files dumped to tape volumes that were produced by the DMF administrative scripts run_full_dump.sh or run_partial_dump.sh. The dmxfsrestore command is a wrapper around xfsrestore(1m) that performs tape volume mounts for the administrator and then executes and turns control of the restore over to xfsrestore.
The dmfind(1) and dmls(1) command source is now based on IRIX the find(1) and ls(1) commands. The man pages have been revised to reflect these changes.
The -p option was added to the dmcopy command. This option lets you have separate invocations of dmcopy to copy data to the same destination file. You must use the option on each invocation. Repeating the option will ensure that a single invocation of dmcopy without -p does not truncate the file at an inopportune time. You should not assume that because you issued one dmcopy first that it will automatically reach the daemon's input queue before the others.
The dmdadm(8), dmvoladm(8), and dmcatadm(8) commands have an additional option, -j. This option disables database journaling. Use it when you are loading an empty database from text records, to prevent the creation of a large and operationally useless journal file. It is not recommended that you use the option in any other application, as it could cause existing journals to become corrupt and unable to accurately recreate the database in the case of a catastrophic loss of the database.
Several key aspects of DMF are changing with release 2.6.3, as documented in previous releases. Before upgrading to this release, please read the following information and be sure that your site is able to accommodate the changes. Failure to do so will cause major problems when the DMF is initialized.
Changes to supported operating systems and features are as follows:
DMF no longer supports the IRIX 6.2 release. It now supports IRIX 6.4.1, 6.5 and later releases. If your site is still running IRIX 6.2, do not upgrade to DMF 2.6.3.
Support for the tape MSP Autoloader API has been dropped. DMF supports both OpenVault and the Tape Management Facility (TMF). If you have not yet acquired OpenVault or TMF, do not upgrade to DMF 2.6.3.
TMF is available to all DMF licensed customers for no charge. If you want TMF, order product SC4-TMF-XL. This will grant TMF, free of charge, to any existing DMF-licensed customers.
DMF is licensed for several different architectures based on the number of CPUs, and DMF does enforce a node locked license. However, it does not enforce licenses based upon the size of the machine.
It is highly likely that a later version of DMF 2.6.3 may start to enforce licenses based upon the number of CPUs in the machine.
Also, the version of FLEXlm being used is likely to change in a future release. This will require SGI to resend all existing DMF customers new license keys.
All existing releases and updates of DMF 2.6.x have been placed on the external public web pages for download. Shortly after release 2.6.3, SGI will remove DMF updates from the public web pages and place the updates in a customer accessible location for DMF licensed customers with DMF support contracts. SGI will leave a recent update on the public page for demonstration and evaluation purposes. SGI plans no change to the frequency of updates, and DMF will be supported in its usual way.
The basic structure of DMF is the same in DMF 2.5 (for UNICOS and UNICOS/mk environments) and DMF 2.6 (for IRIX environments). Changes were made to update the product in areas affected by operating system dependencies. The DMF administrator interface was improved in the areas of database administration utilities and automatic space management. There have also been changes in basic terminology. The changes are briefly described as follows:
The MOUNT_OPTIONS configuration parameter is no longer used. To specify label type and block size, use the LABEL_TYPE and BLOCK_SIZE parameters of the device object. The TMF_TMMNT_OPTIONS parameter allows you to specify a tape group name (the -g option) and to specify how compression is done (the -i option).
DMF 2.6 product installation is very different from the procedures used under DMF 2.5.
Under the UNICOS operating system, the kernel interface that supports file state transitions is dmofrq(2). Under the IRIX operating system, this interface is DMAPI 2.3.
The syntax of the configuration file was revised and all the file calculation information and MSP selection information is now embedded in this file.
The use of the HOME_DIR, SPOOL_DIR, and JOURNAL_DIR directories has been restructured.
Automated space management is now handled entirely by the new program dmfsmon(8); therefore, the fsmon(8) and dmmctl(8) commands were not ported.
In DMF 2.6.2, tape mounting can be accomplished by using OpenVault, the Tape Management Facility (TMF), or a set of internal routines extracted from the UNICOS tape subsystem (DMF refers to these routines as the autoloader). The autoloader is available only temporarily.
Because IRIX does not support the UNICOS user database feature (UDB), the protected files feature of DMF 2.5 is not supported under DMF 2.6.
The dmmode(2) command is not supported under DMF 2.6.
The client/server configuration option is not supported under DMF 2.6.
The new dmscanfs(8) command provides the report functionality of the dmhit(blank) command, which was not ported.
The commands dmdalter and dmdbase have been replaced with a single command, dmdadm(8), which has an administrator interface similar to that of dmcatadm(8) and dmvoladm(8).
The dmmigall command has been renamed dmmigrate(8).
The IRIX ls(1) and find(1) commands were not modified to provide information about DMF managed files. To find this information, use the DMF commands dmls(1) and dmfind(1), which are based on the IRIX commands.
Because IRIX supports sparse files, the file length does not necessarily indicate the actual space occupied by the file on disk. DMF works with the actual space used.
The tape format was modified to be non-system specific and to handle files with sparse data. Tapes written by DMF at 2.6 on IRIX systems cannot be read by DMF on UNICOS systems.
The FTP MSP stores file data differently. It now uses the same format as the tape MSP uses. This allows the FTP MSP to efficiently handle sparse files and provides for enhanced recoverability.
The structure of the directory written by dmsnap(8) is different.
The dmdump(8) and dmsort(8) commands use f77 blocking for binary data.
File handles are now called bit-file identifiers (bfids) and dev/inode is now called fhandle.