Procedure 2-2. Daemon Database Record Length Configuration

The steps to configure the database entry length are as follows:

1. If the dmdaemon is running, use dmdstop(8) to halt processing.

2. If a daemon database already exists, perform the following steps:

   1. cd HOME_DIR/daemon (HOME_DIR is the value of HOME_DIR returned by the dmconfig base command)

   2. dmdump -c . > textfile (textfile is the name of a file that will contain the text representation of the current database)

   3. cp dbrec* pathseg* dmd_db.dbd backup_dir (backup_dir is the name of the directory that will hold the old version of the database)

   4. rm dbrec* pathseg* dmd_db.dbd

3. cd /etc/dmf/dmbase/lib/rdm

4. Backup the dmd_db.dbd and dmd_db.ddl files that reside in /etc/dmf/dmbase/lib/rdm. This will aid in disaster recovery should something go wrong.

5. Edit dmd_db.ddl to set the new path field lengths for the dbrec and/or pathseg records. For the most efficient use of disk space for the dmatmsp, set the dbrec path size to 26.

6. Regenerate the new database definition:

   /etc/dmf/dmbase/etc/dmddlp -drsx dmd_db.ddl

   
   

7. Backup the new versions of dmd_db.dbd and dmd_db.ddl for future reference or disaster recovery.

8. If the daemon database was dumped to text in step 2, complete the following steps:

   1. cd HOME_DIR/daemon

   2. dmdadm -u -c "load textfile" (textfile was created in step 2)

9. If the daemon was running in step 1, restart it by executing dmdaemon(8).

Procedure 2-5. Configuring the daemon_tasks Object

The following steps explain how to define a daemon_tasks object. You can change the object name itself (daemon_tasks) to be any name you like.

Do not change the script names.

You may comment out the RUN_TASK parameters for any tasks you do not want to run.

The following example would configure a daemon_tasks object:

define  daemon_tasks
        TYPE                 taskgroup
        RUN_TASK             $ADMINDIR/run_audit.sh every day \
                                at 23:00
#
        RUN_TASK             $ADMINDIR/run_scan_logs.sh at 00:01
#
        RUN_TASK             $ADMINDIR/run_remove_logs.sh every \
                                day at 1:00
        LOG_RETENTION        4w
#
        RUN_TASK             $ADMINDIR/run_remove_journals.sh every \
                                day at 1:00
        JOURNAL_RETENTION    4w
#
        RUN_TASK             $ADMINDIR/run_copy_databases.sh \
                             every day at 3:00 12:00 21:00
        DATABASE_COPIES      /save/dmf_home /alt/dmf_home
enddef

1. Define the object to have the same name that you provided for the TASK_GROUPS parameter of the daemon object. In the example it is daemon_tasks.

2. Ensure that TYPE is set to taskgroup. There is no default.

3. Configure the RUN_TASK parameters. DMF substitutes $ADMINDIR in the path with the actual etc/admin directory in the configured DMF directory (/etc/dmf/dmbase). When the task is run, it is given the name of the object that requested the task as the first parameter and the name of the task group (in this case daemon_tasks) as the second parameter. The task itself may use the dmconfig(8) command to obtain further parameters from either of these objects.

   All of the RUN_TASK parameters require that you provide a time_expression.

   The time_expression defines when a task should be done. It is a schedule expression that has the following form:

   [every n period] [at hh:mm[:ss] ...] [on day ...]

   
   

   period is one of minute[s], hour[s], day[s], week[s], or month[s].

   n is an integer.

   day is a day of the month (1 through 31) or day of the week (sunday through saturday).

   The following are examples of valid time expressions:

   at 2:00 every 5 minutes at 1:00 on tuesday

   
   

   Some of the tasks defined by the RUN_TASK parameters require more information. The following steps specify what you must provide.

   1. The run_audit.sh task runs dmaudit. For this task, provide a time_expression. If it detects any errors, the run_audit.sh task mails the errors to the email address defined by the ADMIN_EMAIL parameter of the base object (described in “Configuring the Base Object”).

   2. The run_scan_logs.sh task scans the DMF log files for errors. For this task, provide a time_expression. If the task finds any errors, it sends email to the email address defined by the ADMIN_EMAIL parameter of the base object.

   3. The run_remove_logs.sh task removes logs that are older than the value you provide by specifying the LOG_RETENTION parameter. You also provide a time_expression to specify when you want the run_remove_logs.sh to run. In the example, log files more than 4 weeks old are deleted each day at 1:00 A.M. Valid values for LOG_RETENTION are a number followed by m[inutes], h[ours], d[ays], or w[eeks].

      The run_remove_journals.sh task removes journals that are older than the value you provide by specifying the JOURNAL_RETENTION parameter. You also provide a time_expression to specify when you want the run_remove_journal.sh to run. In the example, journal files more than 4 weeks old are deleted each day at 1:00 A.M. Valid values for JOURNAL_RETENTION are a number followed by m[inutes], h[ours], d[ays], or w[eeks].

      ---

      Note: The run_remove_journals.sh and run_remove_logs.sh tasks are not limited to the daemon logs and journals; they also clear the logs and journals for MSP(s).

      ---

      
      

   4. The run_copy_databases.sh task makes a copy of the DMF databases. For this task, in addition to a value for time_expression, provide a value for the DATABASE_COPIES parameter that specifies one or more directories. If you specify multiple directories, breaking the directories among multiple disk devices minimizes the chance of losing all the copies of the database.

      The task copies a snapshot of the current DMF databases to the directory with the oldest copy. Integrity checks are done on the databases before the copy is saved. If the checks fail, the copy is not saved, and the task sends email to the email address defined by the ADMIN_EMAIL parameter of the base object.

Procedure 2-6. Configuring the dump_tasks Object

The following steps explain how to define a dump_tasks object. You can change the object name itself (dump_tasks) to be any name you like.

Do not change the script names.

You may comment out the RUN_TASK parameters for any tasks you do not want to run.

The following example would configure a dump_tasks object:

define  dump_tasks
        TYPE                  taskgroup
        RUN_TASK              $ADMINDIR/run_full_dump.sh on \
                                sunday at 00:01
        RUN_TASK              $ADMINDIR/run_partial_dump.sh on \
                                monday tuesday wednesday thursday \
                                friday saturday at 00:01
        RUN_TASK              $ADMINDIR/run_hard_deletes.sh
                                at 23:00
#
        DUMP_TAPES            HOME_DIR/tapes
        DUMP_RETENTION        4w
        DUMP_DEVICE           SILO_2
        DUMP_MIGRATE_FIRST    yes
        DUMP_INVENTORY_COPY   /save/dump_inventory
enddef

1. Define the object to have the same name that you provided for the TASK_GROUPS parameter of the daemon object. In the example it is dump_tasks.

2. Ensure that TYPE is set to taskgroup. There is no default.

3. Configure the RUN_TASK parameters. See step 3 in Procedure 2-5, for information about $ADMINDIR and time_expression.

   The following steps specify the information you must provide for the tasks to run correctly.

   1. The run_full_dump.sh task runs a full backup of DMF-managed file systems at intervals specified by the time_expression. In the example, the full backup is run each week on Sunday morning one minute after midnight.

   2. The run_partial_dump.sh task backs up only those files in DMF-managed file systems that have changed since the time a full backup was completed. The backups are run at intervals specified by the time_expression. In the example, it is run each day of the week except Sunday, at one minute after midnight.

   3. The run_hard_deletes.sh task removes from the database any files that have been deleted but can no longer be restored because the backup tapes have been recycled (that is, it hard-deletes the files). The backup tapes are recycled at the time interval set by the DUMP_RETENTION parameter described in the next step. For more information on hard-deleting files, see “Soft- and Hard-deletes” in Chapter 7.

   4. Manage the data from the file system dumps by configuring the following parameters:

      DUMP_TAPES DUMP_RETENTION DUMP_DEVICE DUMP_MIGRATE_FIRST DUMP_INVENTORY_COPY DUMP_FILE_SYSTEMS DUMP_VSNS_USED

      
      

      The DUMP_TAPES parameter specifies the path of a file that contains tape volume serial numbers (one per line) for the dump tasks to use.

      The DUMP_RETENTION parameter specifies how long the backups of the file system will be kept before the tapes are reused. This is also the value used by the run_hard_deletes.sh task to determine how old soft-deleted database entries must be before removing them from the database. Valid values for DUMP_RETENTION are a number followed by m[inutes], h[ours], d[ays], or w[eeks].

      The DUMP_DEVICE parameter specifies the name of the device object in the configuration file that defines how to mount the tapes that the dump tasks will use. See “Device Objects”, for information about device objects.

      If you set DUMP_MIGRATE_FIRST to YES, the dmmigrate command is run before the dumps are done to ensure that all migratable files are migrated, thus reducing the tapes needed for the dump. The default is NO.

      The DUMP_INVENTORY_COPY parameter specifies the path name of a directory into which are copied the xfsdump(1M) inventory files for the backed-up file systems.

      The DUMP_FILE_SYSTEMS parameter specifies one or more file systems to dump. If not specified, the task dumps all the file systems configured in the configuration file. Use this parameter only if your site needs different dump policies (such as different dump times) for different file systems. It is safest not to specify a value for this parameter and therefore dump all file systems configured.

      The DUMP_VSNS_USED parameter is optional. It specifies the name of a file to which the tasks that dump the file systems will append the VSN, one per line, of each volume used by xfsdump. If you don't specify this parameter, the task uses /dev/null as the file name.

Procedure 2-8. Configuring Objects for Automated Space Management

The following example defines a policy object for automated space management:

define  fs_space
        TYPE                     policy
        MIGRATION_TARGET         50
        FREE_SPACE_TARGET        10
        FREE_SPACE_MINIMUM       5
        FREE_DUALSTATE_FIRST     off

        AGE_WEIGHT 0    0.00     when age < 10
        AGE_WEIGHT 1    0.01     when age < 30
        AGE_WEIGHT 10   0.05     when age < 120
        AGE_WEIGHT 50   0.1

        SPACE_WEIGHT  0  0
enddef

The following steps explain pertinent information for configuring the above policy object:

1. Ensure that define has a value you set previously in the POLICIES parameter of a filesystem object. There is no default.

2. Ensure that TYPE is set to policy. There is no default.

3. Configure automated space management as follows:

   1. Configure MIGRATION_TARGET to an integer percentage of total file system space. DMF attempts to maintain this percentage as a reserve of space that is free or occupied by dual-state files that can be deleted if the file system free space reaches or falls below FREE_SPACE_MINIMUM. The default is 30.

   2. Configure FREE_SPACE_TARGET to an integer percentage of total file system space. DMF will try to achieve this level of free space when free space reaches or falls below FREE_SPACE_MINIMUM. The default is 20.

   3. Configure FREE_SPACE_MINIMUM to an integer percentage of the total file system space that DMF must maintain as free. DMF will begin to migrate files when the available free space for the configured file system reaches or falls below this percentage value. The default is 10.

   4. Configure FREE_DUALSTATE_FIRST to be on if you want DMF to free the space used by dual-state files before it migrates and frees regular files. The default is off.

4. Configure the age and size weighting factors associated with a file when it is evaluated for migration as follows:

   1. The syntax of the AGE_WEIGHT parameter is a floating-point constant followed by a floating-point multiplier. The age weight is calculated as follows:

      constant + (multiplier x age_in_days)

      
      

      Add a when clause to select which files should use these values. DMF checks each AGE_WEIGHT parameter in turn, in the order they occur in the configuration file. If the when clause is present, DMF determines whether the file matches the criteria in the clause. If no clause is present, a match is assumed. If the file matches the criteria, the file weight is calculated from the parameter values. If they do not match, the next instance of that parameter is examined.

      An AGE_WEIGHT of 1 1.0 is used if no AGE_WEIGHT applies for a file.

      In the example policy, files that have been accessed or modified within the last 10 days have a weight of zero. File migration likelihood increases with the length of time since last access because the file will have a greater weight. The final line specifies that files which have not been accessed or modified in 120 days or more have a far greater weight than all other files.

   2. The syntax of SPACE_WEIGHT parameters is a floating-point constant followed by a floating-point multiplier. The space weight is calculated as follows:

      constant + (multiplier x file_disk_space_in_bytes)

      
      

      In the example policy, the size of the file does not affect migration because all files have SPACE_WEIGHT of zero.

      A SPACE_WEIGHT of 0 0.0 is used if no SPACE_WEIGHT applies for a file.

   3. Configure negative values to ensure that files are never automatically migrated. For example, you might want to set a minimum age for migration. The following parameter specifies that files that have been accessed or modified within 1 day are never automatically migrated:

      AGE_WEIGHT -1 0.0 when age <= 1

      
      The following parameter specifies that small files are never automatically migrated:

      SPACE_WEIGHT -1 0 when space <= 4k

      
      

   ---

   Note: DMF calculates the size weight and age weight separately. If either value is less than zero, the file is not automatically migrated or freed. Otherwise, the two values are summed to form the file's weight.

   ---

   
   

Procedure 2-9. Configuring Objects for MSP Selection

The following example defines a policy object for MSP selection:

define  fs_msp
        TYPE                    policy
        SELECT_MSP none         when space < 65536
        SELECT_MSP cart1 cart2  when gid = 22
        SELECT_MSP cart1        when space >= 50m
        SELECT_MSP cart2
enddef

The following steps explain pertinent information for configuring the above policy object:

---

Note: The space expression references the number of bytes the file occupies on disk, which may be larger or smaller than the length of the file. For example, you might use the following line in a policy: SELECT_MSP none when space < 4096 Your intent would be to restrict files smaller than 4 Kbytes from migrating. However, this line may actually allow files as small as 1 byte to be migrated, because while the amount of data in the file is 1 byte, it will take 1 block to hold that 1 byte. If your file system uses 4-Kbyte blocks, the space used by the file is 4096, and it does not match the policy line. To ensure that files smaller than 4 Kbytes do not migrate, use the following line: SELECT_MSP none when space <= 4096

---

1. Ensure that define has a value that you set previously in the POLICIES parameter of the filesystem object. There is no default.

2. Ensure that TYPE is set to policy. There is no default.

3. Ensure that the MSP name (or names) you specify as the first value of the SELECT_MSP parameter is a name you set previously in the MSP_NAMES parameter of the daemon object. There is no default.

4. Configure MSP selection criteria as follows:

   1. If you want to select an MSP based on file size, use parameters such as the following, which send large files to cart1 and small files to cart2:

      SELECT_MSP cart1 when space >= 50m SELECT_MSP cart2 when space >= 65536

      
      

   2. If you want certain files to be copied to more than one MSP, use syntax such as the following, which migrates all files that have a group ID of 22 to both of the configured MSPs:

      SELECT_MSP cart1 cart2 when gid = 22

      
      Separate multiple MSP names with a blank space.

   3. If you want to ensure that some files are never migrated, you can designate the MSP selection as none. The following line from the sample file ensures that files smaller than 65,536 bytes are not migrated:

      SELECT_MSP none when space < 65536

      
      

Procedure 2-11. Configuring the msp_tasks Object

The following steps explain how to define the msp_tasks object. You can change the object name itself (msp_tasks) to be any name you like.

Do not change the path names or task names.

You may comment out the RUN_TASK parameters for any tasks you do not want to run.

define	msp_tasks
	TYPE			taskgroup
	RUN_TASK	$ADMINDIR/run_tape_report.sh at 00:10
#
	RUN_TASK	$ADMINDIR/run_tape_merge.sh on \
                   monday wednesday friday at 2:00
THRESHOLD          50
#VOLUME_LIMIT      20
#DATA_LIMIT        5g
#
RUN_TASK $ADMINDIR/run_merge_stop.sh at 5:00

1. Define the object to have the same name that you provided for the TASK_GROUPS parameter of the tape msp object. In the example it is msp_tasks.

2. Ensure that TYPE is set to taskgroup. There is no default.

3. Configure the RUN_TASK parameters. DMF substitutes $ADMINDIR in the path with the actual etc/admin directory in the configured DMF directory (/etc/dmf/dmbase). When the task is run, it is given the name of the object that requested the task as the first parameter and the name of the task group (in this case msp_tasks) as the second parameter. The task itself may use the dmconfig(8) command to obtain further parameters from either of these objects.

   The RUN_TASK parameters require that you provide a time_expression.

   The time_expression defines when a task should be done. It is a schedule expression that has the following form:

   [every n period] [at hh:mm[:ss] ...] [on day ...]

   
   

   period is one of minute[s], hour[s], day[s], week[s], or month[s].

   n is an integer.

   day is a day of the month (1 through 31) or day of the week (sunday through saturday).

   The following are examples of valid time expressions:

   at 2:00 every 5 minutes at 1:00 on tuesday

   
   

   The following steps specify the information you must provide for the tasks to run correctly.

   1. The run_tape_report.sh generates a report on the tapes in the MSP tape pool and on MSP activity. In the example, it runs every day at 10 minutes after midnight.

   2. The run_tape_merge.sh task merges sparse tapes. You can specify the criteria that DMF uses to determine that a tape is sparse, as follows:

      • Use the THRESHOLD parameter to set an integer percentage of active data on a tape. DMF will consider a tape to be sparse when it has less than this percentage of data that is still active.

      • Use the VOLUME_LIMIT parameter to set the maximum number of tape volumes that can be selected for merging at one time.

      • Use the DATA_LIMIT parameter to set the maximum amount of data (in bytes) that should be selected for merging at one time.

   3. The run_merge_stop.sh task shuts down volume merging (tape merging) at a time you specify by using a time_expression. This task is an alternative to using the VOLUME_LIMIT and DATA_LIMIT parameters to stop merging at specified points. In the example, the limit parameters are commented out because run_merge_stop.sh is used to control volume merging.

Procedure 2-13. Configuring DMF to Use OpenVault

The following procedure describes how to make OpenVault and DMF work together. The OpenVault setup script can be used to enable the DMF application. See the OpenVault Operator's and Administrator's Guide for a description of this script.

---

Note: The example that follows assumes that before you complete the steps described, the OpenVault server is configured and all drives and libraries are configured and OpenVault is running.

---

1. On the OpenVault server, add DMF as both a privileged and unprivileged OpenVault application for this host. To do this, use the setup script, menu item 1, submenu 5.

2. Add the DMF application as a valid user to appropriate drive groups. It is preferable that you use the OpenVault setup script, menu item 2, submenu 7. If for some reason you cannot use the setup script, you can enter the command manually, as follows:

   ov_drivegroup -a drive_group -A dmf

   
   

3. Add DMF as a valid application to appropriate tape groups. It is preferable that you use the OpenVault setup script, menu item 2, submenu 8. You can enter the command manually, as follows:

   ov_cartgroup -a tape_group -A dmf

   
   

4. Configure the base object for use with OpenVault:

   define base TYPE base HOME_DIR /dmf/home . . . OV_KEY_FILE /dmf/home/ovkeys OV_SERVER hostname enddef

   
   

   1. Configure the OV_KEY_FILE parameter name of the key file that holds security information for OpenVault. It is usually located in HOME_DIR and called ovkeys.

   2. Configure the OV_SERVER parameter to the value returned by the hostname(1) command on the machine on which the OpenVault server is running. This parameter only applies when OpenVault is used as the mounting service. The default value is the host name of the machine on which you are running.

5. Use the dmov_keyfile(8) command to create the file defined by the OV_KEY_FILE parameter. This command will prompt you for the privileged and unprivileged keys that you defined in step 1.

6. Configure the device object for use with OpenVault:

   define timber TYPE device MOUNT_SERVICE openvault OV_ACCESS_MODES readwrite OV_INTERCHANGE_MODES compression ZONE_SIZE 200m enddef

   
   

   1. Ensure that define has a value that you set previously in the TAPE_TYPE parameter of the msp object. There is no default.

   2. Configure TYPE to be device. There is no default.

   3. Configure the MOUNT_SERVICE parameter to be openvault.

   4. Configure the OV_ACCESS_MODES parameter to be a list of access mode names that control how the tape is used. The parameter is optional. The default value is readwrite when migrating and readonly when recalling. Use this parameter to force readwrite.

      The other possible values that OpenVault can use are not configurable in DMF: for rewind/norewind, DMF uses rewind; for variable/fixed, DMF uses variable.

   5. Configure the OV_INTERCHANGE_MODES parameter to be a list of interchange mode names that control how data is written to tape. This can be used to control whether the device compresses data as it is written. This parameter is optional.

      To specify that you want data compressed, use OV_INTERCHANGE_MODES compression

      To force all tapes to be written as DLT4000, use OV_INTERCHANGE_MODES DLT4000

      This parameter is applied when a tape is first used or rewritten.

   6. Configure other parameters relevant to your site. The example sets the ZONE_SIZE parameter to 200 MB. The target zone size is a major factor in determining how much data is written before writing a tape mark and updating the MSP database. Here, the tapes used by the SILO_2 MSP will, in general, have more data written in a zone than DMF uses as a default. Smaller values allow faster recalls and better recovery, but they cause poorer write performance than larger values. The default is 50 MB. See “Media Concepts” in Chapter 6, for more information on how tape zone sizes are determined.

7. Make the appropriate cartridges accessible to the MSPs by assigning the cartridges to the DMF application in OpenVault. To do this, you must know the following:

   • Cartridge type name. To determine the cartridge types allowed by a given drive, enter the following:

     ov_stat -c -D drive | grep base

     
     

     The fourth column shown in the output is the cartridge type.

   • Cartridge group. To determine the possible cartridge groups, enter the following:

     ov_cartgroup -l -A dmf

     
     

   1. If you already have tapes defined in your MSP database, tell OpenVault about these tapes by entering the following:

      dmov_makecarts -g cartgroup -t carttype msp

      
      

   2. If there are unmanaged cartridges in an OpenVault managed library, you can import the unmanaged cartridges, assign them to DMF, and add them to an MSP database by entering the following:

      dmov_loadtapes -l library -g cartgroup -t carttype msp

      
      

      This command will invoke a vi(1) session. In the vi(1) session, delete any cartridges that you do not want added to the MSP.

   3. If neither of the above cases are appropriate, you can manually configure the cartridges. The following commands can be useful in this effort:

      • To list cartridges in a library, enter the following:

        ov_stat -s -L library

        
        

      • To list information on cartridges known to OpenVault, enter the following:

        ov_lscarts -f '.*'

        
        

      • To import cartridges into OpenVault and optionally assign them to DMF use the ov_import command.

      • To assign a cartridge known to OpenVault to an application, use the ov_vol command with the -n option.

Procedure 2-14. Creating MSP Database Records

The following procedure is shown as an example that assumes you have an MSP called cart1.

1. If you have not yet done so, set your PATH environment variable to include /etc/dmf/dmbase/etc. ( See “Setting PATH Environment Variables”.)

2. Enter the following command and it will respond as shown:

   % dmvoladm -m cart1 dmvoladm: at rdm_open - created database atmsp_db adm: 1>

   The response is an informational message indicating that dmvoladm could not open an existing MSP databases, so it is creating a new and empty one. You should get this message the first time you use dmvoladm for an MSP, but never again. The next line is the prompt for dmvoladm directives.

3. Assume that you will use 200 tapes of type CART with standard labels PA0001 through PA0200.

   After the prompt, enter the following directive:

   adm:1> create PA0001-PA0200

4. After entering this directive, you will receive 200 messages, one for each entry created, beginning with the following:

   VSN PA0001 created. VSN PA0002 created.

   
   

5. Use the following dmvoladm directive to list all of the tape VSNs in the newly created library:

   adm:2> list all

   ---

   Note:: The dmvoladm tapesize field is purely for site documentation and is not used by the MSP. The blocksize field documents the value used when the tape is first written or rewritten. It should not be changed in the database; however, if you want another value, change the BLOCK_SIZE nnn configuration parameter of the device object.

   ---

   
   

6. Issue the dmvoladm quit directive to complete setting up the MSP.

   adm:3> quit