Chapter 19. Type Managers and Customized Processing of File Elements

This chapter discusses several features that allow you to classify files, and customize the way in which ClearCase manages different classes of files. These features include:

Scenario

As an example, consider the various kinds of files involved in the monet project, discussed in several earlier chapters. Also see Table 19-1. (For simplicity, we won't attempt to classify all the files, just a representative sample.)

Table 19-1. Files Used in `monet' Project

Kind of Files

Identifying Characteristics

Source Files

 

C-language source file

.c file name suffix

C-language header file

.h file name suffix

FrameMakerTM binary file

<MakerFile> as “magic number”

manual page source file

.1 – .9 file name suffix

Derived Files

 

ar(1) archive (library)

.a file name suffix

compiled executable

<varies with system architecture>


File Typing

In various contexts, ClearCase determines one or more file types for an existing file system object, or for a name to be used for a new object:

  • When a mkelem command is entered without the -eltype option, file typing is performed on the new element's simple file name (leaf name).

  • xclearcase sometimes displays file system objects as icons. To do so, it performs file typing on each object; then, it uses the file type to select an icon.

The file typing routines use predefined and user-defined magic files, as described in the cc.magic manual page. A magic file can use many different techniques to determine a file type, including file name pattern-matching, stat(2) data, and standard UNIX “magic numbers”.

For example, the magic file listed in Figure 19-1 specifies several file types for each kind of file listed in Table 19-1.

Figure 19-1. Sample `Magic' File


Element Types and Type Managers

ClearCase's ability to handle different classes of files differently hinges on the concept of element type. Each file element in a VOB must have an element type. An element gets its type when it is created with the mkelem command; you can change an element's type subsequently, with the chtype command. (An element is an instance of its element type, in the same way that an attribute is an instance of an attribute type, and a version label is an instance of a label type.)

Each element type has an associated type manager, a suite of programs that handle the storage and retrieval of versions from storage pools. Thus, the way in which a particular file element's data is handled involves two correspondences: (1) the file element has an element type; (2) the element type has a type manager. Figure 19-2 shows how these facilities work together.


Note: Each directory element also has an element type. But directory elements do not use type managers — the contents of a directory version are stored in the VOB database itself, not in storage pools.

Figure 19-2. Data Handling: File Type, Element Type, Type Manager


For example, a new element named monet_adm.1 might get its element type as follows:

  1. A user enters a “create element” command:

    % cleartool mkelem monet_adm.1
    

  2. Since the user did not specify an element type (–eltype option), mkelem uses one or more magic files to determine the file type(s) of the specified name. Suppose that the magic file shown in Figure 19-1 is the first (or only) one to be used. In this case, rule (4) is the first one to match the name monet_adm.1, yielding this list of file types:

    manpage src_file text_file file
    

  3. mkelem compares this list with the set of element types defined for the new element's VOB. Suppose that text_file is the first file type that names an existing element type; in this case, monet_adm.1 is created as an element of type text_file.

  4. Data storage and retrieval for versions of element monet_adm.1 will be handled by the type manager associated with the text_file element type; its name is text_file_delta:

    % cleartool describe -type text_file
    element type "text_file"
     01-Feb-93.09:11:32 by VOB administrator (vobadm.dvt@sol)
      "Predefined element type used to represent a text file."
      type manager: text_file_delta
      supertype: file
      meta-type of element: file element
    


Note: ClearCase supports a “search path” facility, using the environment variable MAGIC_PATH. See the cc.magic manual page for details.

File-typing mechanisms are defined on a per-user (or per-site) basis; element types are defined on a per-VOB basis.A new element named monet_adm.1 was created as a text_file element in this case; in a VOB with a different set of element types, the same magic file might have caused it to be created as a src_file element.


Note: Ensuring element-type consistency among VOBs is a manual administrative task.


Other Applications of Element Types

Element types allow differential and customized handling of files beyond the selection of type managers discussed above. Following are some examples.

Using Element Types to Configure a View

Creating all C-language header files as elements of type hdr_file allows flexibility in configuring views. Suppose that one developer has been reorganizing

all of the project's header files, working on a branch named header_reorg to avoid destabilizing others' work. To compile with the new header files, another developer can use a view reconfigured with one additional rule:

element * CHECKEDOUT
element -eltype hdr_file * /main/header_reorg/LATEST
element * /main/LATEST

Processing Files by Element Type

Suppose that a coding-standards program named check_var_names is to be executed on each C-language source file. If all such files have element type c_src, then a single cleartool command does the job:

%  cleartool find -avobs -visible -element 'eltype(c_src)' \	        -exec 'check_var_names $CLEARCASE_PN'

Predefined and User-Defined Element Types

Some of the element types discussed in the sections above (for example, text_file) are predefined. Others (for example, c_src and hdr_file) are not predefined — the examples above work only if user-defined element types with these names are created with the mkeltype command.

When a new VOB is created, it automatically gets a full set of the predefined element types. Each one is associated with one of the type managers provided with ClearCase. The mkeltype manual page describes the predefined element types and their type managers.

When you create a new element type with mkeltype, you must specify an existing element type as its supertype. By default, the new element type uses the same type manager as its supertype; in this case, the only distinction between the new and old types is for the purposes described in “Other Applications of Element Types”. For differential data handling, use the –manager option to create an element type that uses a different type manager from its supertype.

Directory /usr/atria/examples/mkeltype contains shell scripts that create a hierarchy of element types.

Predefined and User-Defined Type Managers

Just as ClearCase has predefined element types, it also has predefined type managers. They are described in the type_manager manual page. Each type manager is implemented as a suite of programs in a subdirectory of /usr/atria/lib/mgrs — the name of the subdirectory is the name of the type manager.

The mkeltype -manager command creates an element type that uses an existing type manager. You can further customize ClearCase by creating totally new type managers (and then creating new element types that use them). Architecturally, type managers are mutually independent. But new type managers can use symbolic links to “inherit” some of the functions of existing ones.

The type_manager manual page describes the basic components of a type manager, and outlines the process of creating a new type manager. File /usr/atria/lib/mgrs/mgr_info.h provides comprehensive information on type managers. We suggest that you familiarize yourself with these sources before proceeding to the following sections, which present an extended example of creating and using a new type manager.

Type Manager for Manual Page Source Files

One kind of file listed in Table 19-1 is “manual page source file” (a file coded in nroff(1) format). A type manager for this kind of file might have these characteristics:

  • stores all versions in compressed form in separate data containers, like the z_whole_copy type manager

  • implements version-comparison (compare method) by diff'ing formatted manual pages instead of the source versions themselves

The basic strategy is to use most of the z_whole_copy type manager's methods. The compare method will use nroff(1) to format the versions before displaying their differences.

Creating the Type Manager Directory

The name mp_mgr (“manual page file”) is appropriate for this type manager. The first step is to create a subdirectory with this name:

# mkdir /usr/atria/lib/mgrs/mp_mgr

Inheriting Methods from Another Type Manager

Most of the mp_mgr methods will be inherited from the z_whole_copy type manager, through symbolic links. You might enter the following commands as the root user in a Bourne shell:

# MP=$ATRIAHOME/lib/mgrs/mp_mgr
# for FILE in create_element create_version construct_version \
      create_branch delete_branches_versions \
      merge xmerge xcompare
> do
> ln -s ../z_whole_copy/$FILE $MP/$FILE
> done
#

Any methods that will not be supported by the new type manager can be omitted from this list. The lack of a symbolic link will cause ClearCase to generate an Unknown Manager Request error.

The following sections describe two of these inherited methods, create_version and construct_version, which can serve as models for user-defined methods. Both are actually implemented as scripts in the same file, /usr/atria/lib/mgrs/z_whole_copy/Zmgr.

The `create_version' Method

The create_version method is invoked when a checkin command is entered. The create_version method of the z_whole_copy type manager:

  • compresses the data in the checked-out version

  • stores the compressed data in a data container located in a source storage pool

  • returns an exit status to the calling process, indicating what to do with the new data container

File /usr/atria/lib/mgrs/mgr_info.h shows what arguments will be passed to the method from the calling program (usually cleartool or xclearcase):

/****************************************************************************
 * create_version
 * Store the data for a new version.
 * Store the version's data in the supplied new container, combining it
 * with the predecessor's data if desired (e.g for incremental deltas).
 *
 * Command line:
 *  create_version create_time new_branch_oid new_ver_oid new_ver_num
 *                new_container_pname pred_branch_oid pred_ver_oid
 *                pred_ver_num pred_container_pname data_pname

The only arguments that require special attention are new_container_pname (5th argument), which specifies the pathname of the new data container, and data_pname (10th argument), which specifies the pathname of the checked-out file.

File /usr/atria/lib/mgrs/mgr_info.sh lists the appropriate exit statuses and provides a symbolic name for the create_version method:

# Any unexpected value is treated as failure
MGR_FAILED=1
# Return Values for store operations
MGR_STORE_KEEP_NEITHER=101
MGR_STORE_KEEP_JUST_OLD=102
MGR_STORE_KEEP_JUST_NEW=103
MGR_STORE_KEEP_BOTH=104
 .
 .
MGR_OP_CREATE_VERSION="create_version"

Figure 19-3 shows the code that implements the create_version method.

Figure 19-3. 'create_version' Method


The Bourne shell allows only nine command-line arguments. The shift 1 in Line 1 discards the first argument (create_time), which is unneeded. Thus, the

pathname of the checked-out version (data_pname), originally the 10th argument, becomes $9.

In Line 6, the contents of data_pname are compressed, then appended to the new, empty data container: new_container_pname, originally the 5th argument, but shifted to become $4. (Lines 2-5 verify that the new data container is, indeed, empty.)

Finally, the exit status of the compress command is checked, and the appropriate value is returned (Lines 7–11). The exit status of the create_version method indicates that both the old data container (which contains the predecessor version) and the new data container (which contains the new version) should be kept.

The `construct_version' Method

An element's construct_version method is invoked when standard UNIX software reads a particular version of the element (unless the contents are already cached in a cleartext storage pool). For example, the construct_version method of element monet_admin.1 is invoked by the view_server when a user enters these commands:

% cp monet_admin.1 /usr/tmp   (read version selected by view)
% cat monet_admin.1@@/main/4       (read a specified version)

It is also invoked during a checkout command, which makes a view-private copy of the most recent version on a branch. The construct_version method of the z_whole_copy type manager:

  • uncompresses the contents of the data container

  • returns an exit status to the calling process, indicating what to do with the new data container

File /usr/lib/lib/mgrs/mgr_info.h shows what arguments will be passed to the method:

/****************************************************************************
 * construct_version
 * Fetch the data for a version.
 * Extract the data for the requested version into the supplied pathname, or
 * return a value indicating that the source container can be used as the
 * cleartext data for the version.
 *
 * Command line:
 *  construct_version source_container_pname data_pname version_oid

File /usr/atria/lib/mgrs/mgr_info.sh lists the appropriate exit statuses and provides a symbolic name for the construct_version method:

# Any unexpected value is treated as failure
MGR_FAILED=1
# Return Values for construct operations
MGR_CONSTRUCT_USE_SRC_CONTAINER=101
MGR_CONSTRUCT_USE_NEW_FILE=102
 .
 .
MGR_OP_CONSTRUCT_VERSION="construct_version"

Figure 19-4 shows the code that implements the construct_version method.

Figure 19-4. 'construct_version' Method


In Line 1, the contents of source_container_pname are uncompressed, then stored in the cleartext container, data_pname. The remaining lines return the appropriate value to the calling process, depending on the success or failure of the uncompress command.

Implementing a New `compare' Method

The compare method is invoked by a cleartool diff command. This method will:

  • format each version using nroff(1), producing a pure-ASCII text file

  • compare the formatted versions, using cleardiff

File /usr/atria/lib/mgrs/mgr_info.h shows what arguments will be passed to the method from cleartool or xclearcase:

/****************************************************************************
 * compare
 * Compare the data for two or more versions.
 * For more information, see man page for cleartool diff.
 *
 * Command line:
 *  compare [-tiny | -window] [-serial | -diff | -parallel] [-columns n]
 *          [pass-through-options] pname pname ...

This listing shows that a user-supplied implementation of the compare method must accept all of the command-line options supported by the ClearCase diff command. Our strategy will be simply to pass the options on to cleardiff, without attempting to interpret them. After all options are processed, the remaining arguments specify the files to be compared.

File /usr/atria/lib/mgrs/mgr_info.sh lists the appropriate exit statuses and provides a symbolic name for the compare method:

# Return Values for COMPARE/MERGE Operations
MGR_COMPARE_NODIFFS=0
MGR_COMPARE_DIFF_OR_ERROR=1
 .
 .
MGR_OP_COMPARE="compare"

The Bourne shell script listed in Figure 19-5 implements the compare method. Implementing the xcompare method as a slight variant of compare is left as an exercise for the reader.

Figure 19-5. Script for `compare' Method


Testing the Type Manager

A new type manager can be tested only by actually using it on some ClearCase host. This process need not be obtrusive. Since the type manager has a new name, no existing element type — and hence, no existing element — uses it automatically. To place the type manager in service, create a new element type, create some test elements of that type, and run some tests.

The following testing sequence continues the mp_mgr example.

Creating a Test Element Type. To make sure that an untested type manager is not used accidentally, associate it with a new element type, manpage_test, whose use is restricted to yourself.

% cleartool mkeltype -nc -supertype compressed_file \ 
         -manager mp_mgr manpage_test 
% cleartool lock -nusers $USER -eltype manpage_test 

Creating and Using a Test Element. These commands create a test element that uses the new type manager, and tests the various data-manipulation methods:

% cd directory-in-test-VOB 
% cleartool checkout -nc .
                              (tests `create_element' method)
% cleartool mkelem -eltype manpage_test -nc -nco test.1
% cleartool checkout -nc test.1
                           (tests `construct_version' method)
% vi test.1                        (edit checked-out version)
% cleartool checkin -c "first" test.1
                             (tests `create_ version' method)
% cleartool checkout -nc test.1
                          (tests `construct_ version' method)
% vi test.1
                                   (edit checked-out version)
% cleartool checkin -c "second" test.1
                             (tests `create_ version' method)
% cleartool diff test.1@@/main/1 test.1@@/main/2
                                     (tests `compare' method)

Installing and Using the Type Manager

After a type manager has been fully tested, you can “make it official” with the following procedure.

Installation. A VOB is a network-wide resource — it can be mounted on any ClearCase host. But a type manager is only a host-wide resource — a separate copy must be installed on each host where ClearCase client programs execute. (It need not be installed on hosts that serve only as repositories for VOBs and/or views.)

To install the type manager on a particular host, create a subdirectory in /usr/atria/lib/mgrs, then populate it with the programs that implement the methods. You might create across-the-network symbolic links to a “master copy” on a server host.

Creation of Element Types. Create one or more element types that use the type manager, just as you did in <Emphasis>Testing the Type Manager (but don't include “test” in the name of the element type!). For example, you might name the element type named manpage or nroff_src.

Conversion of Existing Elements. It is likely that you'll want to have at least a few existing elements use the new type manager. The chtype command does the job:

% cleartool chtype -force manpage pathname(s) 

Permission to change an element's type is restricted to the element's owner, the VOB owner, and the root user.

Revision of Magic Files. If you want the new element type(s) to be used automatically for certain newly created elements, create (or update) a local.magic file in each host's /usr/atria/config/magic directory:

manpage src_file text_file file: -name "*.[1-9]" ;

Public Relations. Advertise the new element type(s) to all ClearCase users, describing the features and benefits of the new type manager. Be sure to include directions on how to gain access to the new functionality automatically (through filenames that match `magic' file rules) and explicitly (with mkelem -eltype).

Icon Usage by GUI Browsers

An xclearcase browser can display file system objects either by-name or graphically. In the latter case, xclearcase selects an icon for each file system object as follows:

  • The object's name or its contents determines a list of file types, as described in “File Typing”.

  • One by one, the file types are compared to the rules in predefined and user-defined icon files, as described in the cc.icon manual page. For example, the file type c_source matches this icon file rule:

    c_source : -icon c ;
    

    As soon as a match is found, the search ends. The token following -icon names the file that contains the icon to be displayed.

  • xclearcase searches for the file, which must be in bitmap(1) format, in directory $HOME/.bitmaps, or /usr/atria/config/ui/bitmaps, or the directories specified by the environment variable BITMAP_PATH.

  • If a valid bitmap file is found, xclearcase displays it; otherwise, the search for an icon continues with the next file type.

The name of an icon file should include a numeric suffix, which need not be specified in the icon file rule. The suffix tells xclearcase how much screen space to allocate for the icon. Each bitmap supplied with ClearCase is stored in a file with a .60 suffix (for example, lib.60), indicating a 60x60 icon. These bitmaps themselves are all 40x40, however; the discrepancy allows for future revisions of xclearcase to annotate the icons.

The following steps show how you can have xclearcase display manual page source files with a customized icon. In accordance with the preceding sections of this chapter, all manual pages will have file type manpage.

  1. Add a rule to your personal magic file (in directory $HOME/.magic) that includes manpage among the file types assigned to all manual page source files:

    manpage src_file text_file file:  -name "*.[1-9]" ;
    

  2. Add a rule to your personal icon file (in directory $HOME/.icon) that maps manpage to a user-defined bitmap file:

    manpage : -icon manual_page_icon ;
    

  3. Create a manpage icon in your personal bitmaps directory ($HOME/.bitmaps) by revising one of the standard icon bitmaps with the standard X bitmap utility:

    % mkdir $HOME/.bitmaps 
    % cd $HOME/.bitmaps 
    % cp $ATRIAHOME/config/ui/bitmaps/c.60 manual_page_icon.60
    % bitmap manual_page_icon.60 
    

  4. Test your work by having an xclearcase graphical directory browser display a manual page source file (Figure 19-6).

    Figure 19-6. User-Defined Icon Displayed by xclearcase