If you want to include, exclude, or sort the order of the columns of data in your dataset before sending it to a classifier or visualizer, you can do this from the Data Transformations panel of Tool Manager. The Add Column Button entry below describes the process of adding a column.

Add Column Button

You can use the Add Column button to create a new column whose values are computed based on a mathematical expression. For example, you could add a new column whose values are the ratio of values from two existing columns. Click Add Column to get a dialog box that lets you specify the new column name and expression (Figure 1-1).

Figure 1-1. Add Column Dialog Box

The left side of this dialog box provides a field for entering the new column's name, and a pulldown menu lets you specify the column type (integer, string, floating point, and so on).

The right-hand side of the dialog contains a large text entry area where you can type in a definition of the expression. As a shortcut to typing column names and operators, scrolled lists in the lower left of the dialog display all columns in the current table and all possible operators. To insert a column name or operator into the expression, either double-click it in its scrolled list, or select it and click the arrow button to the right of the scrolled list.

The expression language used in the Filter and Add Column panels is similar to expressions in C, C++, and Java. The basic operators are the same:

+	addition
-	subtraction
*	multiplication
/	division
( )	parentheses for grouping expressions
%	modulo (remainder after division)
!	logical NOT
~	logical NOT
&&	logical AND
||	logical OR
^	logical exclusive OR
==	equal to
!=	not equal to
<=	less than or equal to
<	less than
>=	greater than or equal to
>	greater than
&	bitwise AND
|	bitwise OR

The expression language also provides the following:

isNull( )	Determines if the value in parentheses is null.
if ( ) then ( ) else ( )	If the value in the first set of parenthesis is true, then the cell is given the value in the second set of parenthesis. Otherwise, the cell is given the value in the last set of parenthesis.
( x ) ? ( y ) : ( z )	C syntax if/then/else
divide( x, y, z )	The cell is given the value of the quotient of x divided by y, unless y is 0, in which case the value is 0.
strlen( x )	The cell is given a value equal to the number of characters in the string.
substring( x, y, z )	The cell is given a substring of string x, that starts at position y, and is of length z. Following the C, C++, and Java convention, the first character in a string has index 0.

To check the expression you have created, click the Check Expression button. If there is an error, a dialog box appears, indicating what the error is and where it occurred. When you click OK, the expression is automatically checked, and the dialog box is not removed unless the expression is correct.

The Add Column dialog box checks for type compatibility: if you have assigned a numerical expression to a string column (or vice versa), a warning message appears, and the type of the new column is automatically changed to be correct.

You can add your own user-defined functions as well. See the MineSet Enterprise Edition Interface Guide .

Using the Tool Manager's Aggregate button requires a basic understanding of arrays and distribution. See “Arrays” for a basic introduction of concepts used in the aggregation feature.

You can use the Tool Manager's Aggregate button to create simple aggregations, make arrays, or distribute columns. Clicking this button causes the Aggregate dialog box to appear (Figure 1-2). It shows three lists, with the columns in the current table appearing in the middle list.

To aggregate, select the name of the column, and click the left arrow button between the left and center lists. The popup menus below let you specify indexes (if the result is to be an array) and a distribution column (if the result is to be distributed).

Five check boxes at the bottom let you specify how different values are to be combined when aggregated: they can be summed or averaged or the minimum value, maximum value, or count (number of records) can be used. When aggregating number-valued columns, you can choose any combination of these options. For other types, only count is permitted. If you choose more than one option, you get more than one result. For example, selecting average and max gives you one result with average values, and another one with maximum values. The check box “Include nulls in aggregation” allows you to either ignore or include null values in the calculations.

Figure 1-2. Aggregate Dialog Box

The three lists of column names are given below:

After you have finished with the additional aggregate criteria dialog box, the Current Columns text box in the Data Transformations pane of the Tool Manager window shows the new column names that result from applying these criteria.

The menus in the lower right corner of the dialog box are used to create distributed columns or array columns. These indexing and distribution menus allow you to specify an array index or distributed column. See “Arrays” and “Distributed Columns”.

The Scatter Visualizer, Splat Visualizer and Map Visualizer offer the capability of animation, provided that the dataset being used has at least one slider element mapped to a column. This means you can use animation to display changes in a dataset over a dimension such as time. Typically, independent attributes, such as time or age, make the best slider dimensions, but any binned column can be used.

The animation control panel, which appears to the right of the main visualizer window, consists of a summary window, with up to two adjacent sliders, an information field, animation buttons, the path slider, the speed slider, a data point toggle button, a synchronize sliders button, and for the Scatter Visualizer, a tube trails menu.

Sliders Controlling Independent Dimensions

The number of sliders appearing adjacent to the summary window is dependent on the dataset displayed in the Visualizer's main window. Datasets can have two, one, or no slider dimensions.

Datasets With Two Independent Dimensions

If the dataset has two dimensions of independently varying data (such as the data set in company.scatterviz), the controls to the right of the main window become visible (see Figure 1-3).

Figure 1-3. Animation Control Panel With Summary Window and Both Slider Controls

To the right of the main window are the summary window and slider controls. The summary window has a horizontal slider below it for selecting data points of the first independent dimension, and a vertical slider to the left for selecting data points of the second independent dimension. The horizontal slider's dimension is identified by a label below it. The vertical slider's dimension is identified by a label above it.

Sliders may be specified in the Tool Manager either by:

• Mapping column names to the Slider 1 and Slider 2 elements in the Data Destination pane for the Map, Scatter, or Splat Visualizers, or

• Creating one or two array columns using the Aggregation panel. These array columns are automatically used for the visualization tool animation sliders.

The summary window shows the total values of the column mapped to the summary visual element for all possible settings of the animation sliders. That is, it shows how the summary attribute varies over the slider dimensions. The whiter the areas of the summary window, the lower the total values represented by the entities in the main window. The denser the color, the higher the values.

The summary window also shows black dots, evenly spaced across the one or two dimensions of data. These dots indicate the precise positions of the discrete datapoints. You can turn off these black dots using the Show Data Points check box.

For example, when the company.scatterviz file is first opened, the 2D summary window shows a color range from white (on the left) to red (on the right). White corresponds to a low sales volume; red represents a higher aggregate sales volume. In this example, the greater the density of red, the higher the total sales of life, auto, and home insurance.

An animation path is a path between the summary window's black dots over which the data may be animated. There are three ways to create an animation path in the summary window.

Open the file, and create an animation path in any of these ways:

The VCR-like buttons and sliders (Path and Speed) below the 2D summary window let you control the animation.

The Apply Model button in the Data Transformations panel of Tool Manager lets you:

On the upper left of this dialog box (Figure 1-4) is a list of all classifiers currently available on the server. If you select a classifier, the right-hand side lists the column names and types required by that classifier. If these requirements match the current table, a message at the bottom states this. If the current table does not have all the columns required for the selected classifier, the message at the bottom states this, the columns that are missing are selected in the list on the right, and the button on the bottom is deactivated.

Figure 1-4. Apply Model Dialog Box: Selecting a Classifier

Apply Model Panel

The Apply Model panel is used to apply a previously created classifier to the current table, as shown in Figure 1-5. There are two modes of application for the classifier:

• Predict label values  for the records in the current table. For example, if you created a classifier to determine churn, you can use this option to add a column that labels each customer as either likely to churn or not likely to churn.

• Estimated probability values  for label value. Instead of using the classifier to predict the label value of each record, it is used to estimate the probability that each record has a specified label value (for example, churn = yes). Given the classifier created to determine churn, you can use this option to add a column that indicates the probability that each customer is likely to churn.

The New column name text field lets you specify the name of the new column.

Figure 1-5. The Apply Model Panel

After building a predictive model, you can apply it to records to predict their label. For example, if you built a classifier (one type of predictive model for discrete labels), for predicting iris type, you can apply the classifier to records containing only the descriptive attributes, and a new column is added with the predicted iris type.

To ensure data quality, after building a classifier you can apply it to the training set in order to identify records that are mislabeled by the classifier. Such records might warrant closer investigation. Perhaps they are “noise,” or they might yield special insights.

For example, suppose a Decision Tree for the iris dataset was induced using the Classify Only mode. If you applied the classifier to the dataset, you get a new column (iris type_1) containing the predicted labels. You can then add a column that is defined as type int with the expression (iris type != iris type_1). The new column has a 1 whenever the classifier misclassifies, and a zero when it correctly classifies. Figure 1-6 shows a Scatter Visualizer plot of the data where the new column is mapped to color with the colors set such that green is 0 (OK) and 1 is red (error). By looking at the plot, it is possible to determine where mistakes are being made.

Figure 1-6. Iris Dataset Misclassification, Example 1

Another alternative is to define the new column as a float with the expression (iris type != iris type_1) + 0.01. The Scatter Visualizer can then be used with the original label mapped to color, and this new column mapped to size. Incorrect predictions are shown as big cubes; correct predictions are shown as small cubes (see Figure 1-7).

Figure 1-7. Iris Dataset Misclassification, Example 2

For further information on errors and misclassification, see “Error Estimation”.

Association rules let you mine data by constructing, verifying, and graphically representing models of patterns in large databases. These patterns are expressed by rules of association, which indicate how often column values occur together in a dataset. A typical application of association rules is market basket analysis, which means identifying which items tend to be found together in a shopping market basket.

Discovering and graphically displaying association rules can be relevant to many enterprises. Some examples of where association rules may generate useful associations are supermarket inventory planning, shelf planning, and attached mailing in direct marketing.

There are two steps involved in working with association rules:

Association Rules can generate both simple (one-to-one) and multiway rules of association. This section describes simple association rules. For a description of multiway rules, see “Multiway Rules ”.

A simple association rule states that given that X is true, there is a certain probability that Y is also true. MineSet refers to X as the left-hand side (LHS) of the rule and Y as the right-hand side of the rule (RHS).

One example of applying association rules is to obtain market basket data for customer buying patterns. Here, a market basket is the set of items bought by a customer on a single visit to a store. An example rule in this context might be: “80% of the people who buy diapers also buy baby powder.” This percentage is known as the confidence of the rule.

In this example, “diapers” is the item on the left-hand side (LHS) of the rule, and “baby powder” is the item on the right-hand side (RHS) of the rule.

Some applications of these rules are:

The Association Rules Generator processes an input file, then generates an output file consisting of the rules. If X and Y are attributes in a record, then a rule such as:

X=>Y 

indicates that whenever X occurs in a record, expect Y to occur with some frequency.

The strength of the association is quantified by four numbers:

The Association Rules Generator does not report rules in which the confidence is less than the expected confidence. In other words, a rule such as A=>B is not reported if the frequency of A and B occurring together is less than the frequency of B alone.

---

Note: Given just Y and a rule of the form X=>Y, nothing is known about X. Rules specify implications only from the LHS to the RHS.

---

The Assoc. tab is the Mining Tools selection in the Data Destinations pane of Tool Manager from which you generate Association Rules. These rules, often called “market basket analysis,” can help you identify common groupings of occurrences.

You can configure the components of association rules using the Tool Manager which greatly simplifies the task. To configure Association Rules from the Tool Manager Data Destinations pane, select Mining Tools, and click the Assoc. (Associations) tab. Depending on the data source, you may want to bin or remove some columns to simplify your visualization.

Association Rules Options

The Association Rules dialog lets you specify several options.

Confidence

This option lets you quantify the number of records in which both sides of the rule appear, divided by the number of records in which the LHS rule appears. For example, if the confidence is 50%, Y occurs in 50% of the records in which X occurs. The minimum confidence threshold default is 50%.

Support

This option lets you quantify how prevalent the rule is throughout the dataset, or how often X and Y occur together in the dataset as a fraction of the total number of records. For example, if the support is 1%, X and Y occur together in 1% of the total number of records. You can specify a minimum support threshold for the generated rules. The default minimum support threshold is 1%.

Weight

Association rules allow for record weighting for those cases in which you want to specify that certain records are more important than others or when you want to compensate for uneven sampling. If Use Weight is not checked, then each record has a weight of one. When the box is checked, a menu allows you to choose the column which contains the weight for each record. The Weight is attribute box, if checked, includes the weight column in the rules found by the Association Rule Generator. If the box is unchecked, the weight column will be excluded from any rules found by the Generator. See “Record Weighting” for a further explanation.

Multiway Rules

If you check Multiway Rules button, the Association Rule Generator generates all rules which satisfy the minimum support and confidence thresholds, including those that have more than one item in the LHS and RHS. An example of such a rule might be “beer and linguini implies potato chips and salsa and wine.”

Multiway association rules generate rules with multiple items on the LHS and/or the RHS. Figure 1-8 illustrates the Tool Manager Association panel configured for multiway rules generation. The version shown is from MineSet for Windows. The IRIX version is similar.

Figure 1-8. Initial Tool Manager Window Showing Multiway Association Generation

Multiway rules are displayed using the Record Viewer rather than the Scatter Visualizer. They are displayed with one rule per row. The first two columns of the table contain the number of items in the LHS and RHS. The next four columns contain the support, confidence, expected confidence, and lift values. The last two columns contain the LHS and RHS items. In the LHS and RHS columns, the items are separated by the word “and.” In the example rule above, the LHS contains two items and is represented as “beer and linguini.” The RHS contains three items and is represented as “potato chips and salsa and wine.”

You can limit the size of the rules generated by entering a number in the “Max total items per rule” field. This number indicates the maximum number of items that are allowed in any rule. The number of items in a rule is the sum of the number of items in the LHS and RHS. The example rule above has five items; simple rules have two items.

---

Note: Generating multiway rules can take a long time. Watch the status window for an indication of the number of rules generated at each iteration. If too many rules are being generated, cancel the operation and increase the minimum support or confidence thresholds, or decrease the maximum allowable number of items per rule.

---

Association rules let you map attributes of the rules to visual elements of the display. Specific mapping allows you to test hypotheses for greater understanding of the visualization. From the Data Destinations pane of Tool Manager, clicking on the RuleViz Mappings button opens the Association Rules Mappings panel shown in Figure 1-9. Each element shows only the appropriate selections in the popup menu.

Figure 1-9. Association Rules Mappings Panel

You can map the columns from the automatically generated .rules.data file (support, confidence, expected confidence, and lift) to visual elements in this panel; defaults are shown in Table 1-8. Refer to “Association Rules Options” for an explanation of the values:

Association rules are displayed graphically to permit you to explore and compare the generated rules. The rules are presented on a grid landscape using the Scatter Visualizer. The left-hand side (LHS) items are on one axis, and right-hand side (RHS) items are on the other. Attributes or characteristics of a rule are displayed at the junction of its LHS and RHS item. The display can include bars, disks, and labels.

If the displayed view is too small, item labels do not appear on the sides of the axes. You can zoom in on the view using the Dolly wheel until the item labels appear. You can also view the labels for a particular rule by placing the mouse pointer over an individual bar when the mouse is in select mode. All of the details for that particular rule will be displayed in the upper left-hand corner of the view area.

A legend indicating the mapping between displayed attributes (such as bar heights and colors) and the values associated with the underlying rules (such as confidence and support) is displayed at the bottom of the main window.

Bar heights correspond to support and bar colors correspond to lift. When the scene is zoomed in enough, the LHS and RHS axes are labeled with the item names, unless this has been turned off in the configuration file.

You can change the labels as well as what the heights and colors of the bars and disks represent by modifying the configuration file using the Tool Manager (see Table 1-8), or by using an editor to change the configuration file. Color maps are automatically produced when a variable is mapped to disk or bar color. If you wish to change these default color maps, you can edit the configuration file.

If you place the cursor over a bar representing an Association Rules object, then click the left mouse button, the information appears in the Selection Window. Multiple rules may be selected by holding down the Shift key while clicking.

Drilling through is the term used to refer to the act of looking at the underlying data in a visualization. The drill through expression is determined by doing a logical “and” of selected rules. Since the columns in the original table do not match the columns in the .rules.data file, the rules Generator produces a special column to help construct the filter expression when a drill through is performed. This means that changing the drill through preferences panel has no effect, because a special string-valued column has already been mapped to drill through in the .rules.scatterviz file.

When you drill through on a rule or rules, MineSet shows all the records that satisfy the rule(s). From the visualizer Selection pulldown menu, choose Show Original Data. See “Drill Through” entry for more information about drilling through.

Converting Files from .ruleviz to .scatterviz

Prior to MineSet 2.6, association rules were displayed in their own visualizer, and the configuration files had a slightly different format.

If you have existing .ruleviz files you can convert them to .scatterviz format by editing the existing .ruleviz file and saving it as a .scatterviz file. Example 1-1 and Example 1-2 show the differences between the .ruleviz and .scatterviz formats. Example 1-2 has embedded comments to help you with the changes. Both configuration files use the same data file.

---

Note: In the old ruleviz file format, size was called height, confidence was called predictability, and support was called prevalence.

---

Example 1-1. group.ruleviz

MineSet 2.5 input { file “group.rules”; } expressions { double `pred/expected`= predictability/expected; } view { height predictability; height max 10; height legend on; disk height expected; disk height legend label “disk height: expected predictability”; color prevalence; color colors “white” “purple”; color scale 0 10; color legend “0%” “10%”; message “%s implies %s\npredictability: %.2f predictability/expected: %.2f prevalence: %.2f”, LHS, RHS, predictability, `pred/expected`, prevalence; options grid size 3; options hide disk distance 600; options hide item distance 600; }

Example 1-2. group.rules.scatterviz

MineSet input { # Rename group.rules to group.rules.data: file “group.rules.data”; # The schema for the rules.data file is always # the following. Add these lines: int nlhs; int nrhs; float support; float confidence; float `expected confidence`; string LHS; string RHS; } expressions { float lift = confidence / `expected confidence`; } view { # This replaces height predictability: size confidence, scale 1.; size legend label “Bar Height: confidence”; # This replaces disk height expected: disk height `expected confidence`, scale 1.; disk height legend label “Disk Height: expected confidence”; # This replaces color prevalence: color support; color colors “white” “purple”, legend label “Color: support”; color scale 0 9; color legend “0%” “9%”; # Add these two axis mappings (not present in old file): axis RHS, max 100, orderby alpha; axis LHS, max 100, orderby alpha; # Make sure the shape type is bar: options entity shape bar; options axis label size 20; message “%s implies %s\n support=%2.2f%%, confidence=%2.2f%%, expected confidence %2.2f%%, lift=%2.2f”,LHS, RHS, support, confidence, `expected confidence`, lift; options grid color “#202020”; options hide disk distance 600; options hide entity label distance 600; }

The Tool Manager Bin Columns panel gives you the option of selecting Automatic Binning. If you select the columns you want binned and then select Automatic Binning, on Windows, or “Automatically choose number of bins” from the Automatic Thresholds tab on IRIX, MineSet will do the binning for you. This is useful when you want the program to use machine learning to suggest bins. See “Binning” for a full description of binning options. Table 1-9 lists the available automatic binning options:

During automatic binning, thresholds are chosen so that the distributions of labels within different bins are as different as possible. This approach continues to create thresholds that split the range until no additional interval is considered significant. The more distinct values, the more bins are chosen (the relationship is logarithmic).

Clicking the Auto check box tells MineSet to determine the minimum weight automatically based on the total weight of the instances: the more total weight, the higher the minimum weight per bin (the relationship is logarithmic).

Backfitting allows you to build a structure from a training set, then backfit a large dataset to improve the probability estimates. The purpose of backfitting is to make the model's probability estimates consistent with the underlying data. Backfitting is faster than inducing the structure of a large model. When you use holdout error estimation, you leave out a portion of the data for testing. When you backfit all the data through the model structure, the error of the final model is reduced, because counts, weights and probabilities reflect the entire dataset. You can find backfitting in the Advanced Options panel of any inducer.

The model that the inducer builds has two parts:

Backfitting a model with a set of records does not alter the structure of the model, but updates the probability estimates based on the new data. Backfitting is useful for several reasons:

When using the drill-through function from the visualizers, you can see data corresponding to the weights shown, which reflect the whole dataset. If backfitting is not used, the weights shown represent only the training set.

You can access “ Backfit test set” from the Data Destination panel's Advanced Options button (Further Options for IRIX) for all inducers. In the Data Destination pane click Mining Tools and select the Classify or Regress tab, and choose Classifier (or Regressor) and Error mode for any inducer. The backfit checkmark is disabled when Boosting is enabled.

Running MineSet in batch mode allows the Tool Manager to perform the modeling calculations without bringing up any visualizations. Batch mode can be particularly useful in projects requiring lengthy computations that need to be done frequently. For instance, the computations can be run at night so the data will be ready the next morning. For details on using batch mode, see MineSet Enterprise Edition Interface Guide.

Binning lets you sort the information from one or more columns into groups, creating a new column in the process (for example, one with a range of ages, 0-18, 19-25, 26-35, and so on). Binning in Tool Manager saves computation time and prepares a simpler visualization. For details on how to bin columns, see “Changing or Creating New Bins for Columns” in the MineSet Enterprise Edition User's Guide for Windows .

(lower-threshold ... upper-threshold]

Boosting is an algorithm that creates several different models and combines their predictions using a weighted voting scheme. Boosted models often improve accuracy by focusing the induction process on examples in the data which are harder to model than others. Boosting is enabled with the Tool Manager from the Further Inducer Options dialog box in Decision Tree and Evidence inducer modes. Select the Boost (no viz) check box.

In some cases, the most important issue in creating a model is error rate. For example, suppose you have analyzed a dataset for churn prediction to a point that you are satisfied, and are ready to create a model that will predict which of your customers are the most likely to churn. At this point, you are no longer interested in visualizing your model, since you have a reasonably good understanding of the factors that are involved. You also want to achieve the best classification accuracy possible. In this case, you might want to enable boosting.

Boosting will not always increase accuracy, but it often does. Boosted models cannot be visualized, though you can still see confusion matrices, lift curves, learning curves and ROI curves for them. Boosting is a computationally intensive process, often taking 25 times longer to run than the corresponding inducer without boosting. Backfitting does not work with boosted classifiers, because of the special way boosting weights multiple models and the records used to train them.

You can use boosting with labels that have any number of values. Boosting will not always improve the error rate of induced models; this is especially stressed for problems that have more than two label values.

Brushing is a feature that allows you to select the same data points in more than one visualization at a time. When you select something in a visualization that supports the sending of brushing events, it will also become selected in any other visualization that is currently open and that supports the receiving of brushing events. Scatter Visualizer and Evidence visualizer support both send and receive brushing events. Tree Visualizer, Map Visualizer and Decision Table Visualizer support only the sending of brushing events.

This Tool Manger option allows you to change the type or name of a column. See the MineSet Enterprise Edition User's Guide for Windows  for details.

Choice point is an advanced option shown when you click Iterative k-means from the Cluster tab of the Tool Manager Mining Tools. The choice point is a value between zero and one which guides, for instance, the selection of the number of clusters; higher choice points suggest a larger number of clusters while lower choice points suggest smaller. A choice point of 1.0 will always pick the upper boundary. In clustering, if your boundaries are one cluster and five clusters, a choice point of 0.4 might pick 2 clusters while a choice point of 0.8 will pick four. A choice point of 1.0 always picks five.

A classifier predicts one attribute of a set of data, given several other attributes. A classifier is a type of model. An attribute is an inherent characteristic in the dataset. For example, if you have data on customers of a telecommunications company, a classification model can be generated to predict whether the customer will churn (leave the company) or not, given information such as whether the customer has voice mail, an international plan or not, and how much time they spend on the phone. The attribute being predicted is called the label, and the attributes used for prediction are called the descriptive attributes.

MineSet can build a classifier automatically from a training set. The training set consists of records in the data for which the label has been supplied, (see also “Training Set”). For example, you supply a database table with one column for each descriptive attribute (such as the presence of a voice mail plan, the average number of calling minutes per day), and one column for the label (churned or not). An algorithm that automatically builds a classifier from a training set is called an inducer.

When a classifier is generated, MineSet also generates a visualization that can help you understand how the classifier operates. This visualization can also provide valuable insight into the data itself. Once a classifier is generated, it can be used to classify records where the label attribute is unknown. This value is predicted by the classifier.

The classifier has two parts: structure and probability estimates

A generated classifier is named with the prefix of the session filename, as determined in Tool Manager, and the appropriate suffix, for instance, -dtable.class for Decision Table classifiers, -dt.class for Decision Tree classifiers.By default, all classifiers are stored on the server in the file_cache directory, which defaults to mineset_files. These classifiers can be used for future classification of unlabeled records; that is, they can be used to predict the labels for unlabeled datasets (see “Apply Model” and “Backfitting”).

To access the range of MineSet classifiers, from the Data Destination pane of the Tool Manager, click the Mining Tools tab to find the Classify tab. You have a choice of four modes: Classifier and Error, Classifier Only, Estimate Error and Learning Curve. Each of these modes can be combined with any of the various inducers: Decision Tree, Option Tree, Evidence and Decision Table. For details on each inducer see the appropriate entry in this guide. For directions in using these classifiers, see the MineSet Enterprise Edition User's Guide for Windows .

The Cluster tab gives you access to MineSet's Clustering, a useful mining tool for exploring an unfamiliar dataset. Because clustering is a descriptive mining task similar to the discovery of association rules, you do not need to designate a specific column as a label. In addition, the dataset is never split into training and test sets; clustering models are always built from and evaluated on the full dataset.

A clustering model is stored as a set of prototypical records, one per cluster. These represent a weighted average of all data in the cluster and are known as Cluster centers or centroids. Unlike standard database records, cluster centers maintain a distribution for each column in the form of summary statistics for numerical columns or histograms for categorical columns.

Clustering is run from the Mining Tools tab in the Data Destinations pane of the Tool Manager. The purpose of clustering is to determine what if any characteristics in the dataset are similar. You then look at the resulting clusters and experiment with different parameters.

All clustering in MineSet uses a combinatorial algorithm based on the k-means objective function; the algorithm forms clusters by grouping similar records together, aiming to maximize the overall similarity within each group.

To reach the clustering tool, select the Mining Tools tab in the Data Destination panel of the Tool Manager main window. From the subsequent tabs select Cluster. This will expose the main clustering panel.

You can begin the clustering process by clicking Go from the main Cluster panel; there are no required options. By default, you will use the single k-means clustering method to discover three clusters in the data. Once the clustering is complete, you will see an evaluation of the clustering, and then the Cluster Visualizer will appear.

The Cluster panel provides the following options:

Result of clustering:
-----------------------------------------
Overall root-mean-squared distance from record to centroid: 0.216 +- 0.0928 RMS distance from records to each centroid:
Cluster 1: 0.2306 +- 0.09484
Cluster 2: 0.1921 +- 0.09932
Cluster 3: 0.2247 +- 0.08058
Model saved as iris.cluster

You can access the Clustering Options dialog box from the Mining Tools tab of the Tool Manager. Select the Cluster tab, then select the Further options button to get the Clustering Options dialog box. The options presented depend on whether you have accessed the single or iterative k-means method.

The Cluster Visualizer presents a series of box charts and histograms. Once the clustering operation has been run, you can view the cluster centers directly using the Cluster Visualizer.

Alternatively, you can use the apply model feature (see “Apply Model”) directly on the training data to assign a cluster to each record. You can then use many of the other tools in MineSet to explore the resulting clusters.

Many of the tool option dialogs have options for choosing colors. When using the Tool Manager, click the Tool options button (Windows) or Further options (IRIX) button to get the tool option dialog box.

Choosing Colors with the Color Chooser (Windows)

To display a Color Chooser, click on a color swatch or on the color list + sign in a Tool Options panel. If a list of color swatches is to be chosen, the list of swatches appears as in Figure 1-10 (these can be empty initially).

Figure 1-10. Multiple Colors Swatches

When you click on a new color swatch you will see that color reflected in the Recent and Preview areas of the Color Chooser (Figure 1-11). If you change your mind, return to the previous color swatch shown in the Recent grid. Once you have selected a color, click OK, and that color is added to your color list. You can add several colors without dismissing the Color Chooser.

Figure 1-11. Color Chooser Dialog Box

You can also pick a color by using the HSB (hue, saturation, and brightness) or RGB (red, green, and blue) panes of the Color Chooser. In the HSB pane, click in the large colored square. A white circle will appear.

When the H radio button is selected, you can adjust the saturation and brightness by dragging the circle around in the box. To adjust the hue, move the slider next to the rainbow-colored bar.

When the S radio button is selected, you can adjust the hue and brightness by dragging the circle around in the box. To adjust the saturation, move the slider next to the colored bar.

When the B radio button is selected, you can adjust the hue and saturation by dragging the circle around in the box. To adjust the brightness, move the slider next to the colored bar.

Figure 1-12. HSB Pane of the Color Chooser Dialog Box

In the RGB pane, you can use the sliders to pick specific values for the amount of red, green, and blue in your chosen color.

Figure 1-13. RGB Pane of the Color Chooser Dialog Box

Choosing Colors with the Color Browser (IRIX)

If the Color Browser is available, it appears on the tool option panel. MineSet has a color list chooser that uses color swatches. This section describes how to choose, apply, and change color options for the MineSet visualizers.

If only one color is to be chosen (for example a grid color), a single color swatch appears.

Clicking the swatch brings up a Color Browser that lets you change the color of that swatch (Figure 1-15).

If a list of color swatches is to be chosen, a list of swatches appears (these can be empty initially), as shown in Figure 1-14.

Figure 1-14. Multiple Colors Swatches

To edit the color, click a swatch with the left mouse button. This also selects the swatch for making changes to the colors with the buttons. If you click on the swatch with the middle mouse button, the swatch is selected, but the color chooser does not appear.

Next to the list of swatches are four buttons. First is a button labeled with a plus sign (+), which adds a new color at the end of the list. A swatch is added, and the color chooser appears, where you can select the color of that swatch. This Add button is disabled if the maximum number of colors is already in the list.

Next is a button labeled with a minus sign (-). This button deletes the selected color. It is disabled if no swatch is selected, or if the list already has the minimum number of colors.

Next to this Delete button are two buttons to shift the selected color right and left. These buttons are disabled if no swatch is selected, or if the swatch is already at the end of the list.

If there are more colors in the list than room to display them, scroll arrows are added at each end of the list. If the hardware runs out of colors, the color swatches are replaced with text labels showing the color in hexadecimal notation.

The Color Browser (Figure 1-15) appears when you click a color swatch or the add button in the Colors panel of the visualizer's Configuration Options panel.

Figure 1-15. Color Browser (IRIX)

To select a color using the Color Browser:

1. Move your mouse cursor on top of the small white circle in the colored hexagon.

2. Press the left mouse button, and move your mouse around the hexagon. The color beneath the small circle appears in the rectangle next to the Current Color label. This rectangle acts as your color palette while you choose a color.

3. Release the mouse button when the small circle is on top of a color you want. The selected swatch immediately takes on the chosen color.

   You can edit several colors without dismissing the Color Browser; clicking any color swatch in the options panel lets you edit that color in the already posted Color Browser.

Click the OK button when you decide on a color. The Color Browser window closes.

The Col.Imp. tab of the Tool Manager gives access to the Column Importance tool. See “Column Importance ” below.

Column Importance is run from the tab labeled Col. Imp. on the Mining Tools panel of Tool Manager. It helps you discover which are the most important columns in a dataset in discriminating different values for a label column you choose.

The difference between column importance and other data mining tools such as clustering (discussed in “Clustering”) is that with column importance, you decide which label you will use to determine the importance of columns. In clustering, the data itself shows you which are the discriminating factors. Because of the differences in representation for classification models, different attributes may be judged more important for different models. A sample file provided with MineSet, showing a case in which Column Importance might be useful is discussed in Appendix A, “Sample Configuration and Data Files”.

See “Adding Columns”, “Remove Columns”, and “Sorting Column Names”.

Each visualizer, as well as MineSet itself, can be started from the command line, by typing the tool name at the prompt, for example:

On Windows systems type:

Viz [configFile]

For UNIX systems type (for example):

scatterviz [configFile]

Identifying the configuration file (configFile) is optional, but if none is specified, the tool starts with only the File and Help menus operational. You must then specify a configuration file using File > Open.

On UNIX, the various tools use the commands: clusterviz, eviviz, scatterviz, splatviz, statviz, treeviz, and mapviz. Association rules uses a two-part command covered under “Association Rules”.

MineSet requires two files be present for each tool:

You create this configuration file automatically when you use the Tool Manager to specify the options and parameters of the classifier or visualizer you plan to use. Alternatively you can create configuration files for each tool using an ascii text editor. For example, open Word Pad or similar text editor and select “Files of Type: All Documents (*.*)” You can open any .schema and .data file this way. Details and examples can be found in the MineSet Enterprise Edition Interface Guide .

Confusion matrices give a more detailed picture of the errors made by a classifier. Instead of simply analyzing the number of correct and incorrect predictions, the confusion matrix shows the type of errors being made. Displaying a Confusion Matrix is an Advanced option for all classifiers (choose the Classify tab) under the Mining Tools tab of the Tool Manager. Figure 1-16 shows a confusion matrix for a Decision Tree that was induced on the iris dataset.

Figure 1-16. Confusion Matrix for Iris Dataset

The two axes represent:

Entries on the diagonal are correct predictions. Entries off the diagonal indicate incorrect predictions. This representation shows that iris-versicolor and iris-virginica are frequently confused, but iris-setosa is always predicted correctly.

When the cost of making different types of mistakes is uneven, it is frequently useful to influence the type of errors that are being made by using a Loss Matrix (see “Loss Matrix”).

---

Note: The confusion matrix shows the errors made on the test set; thus, it represents the expected true distribution of errors in an actual situation if the underlying distribution of the data does not change significantly. The confusion matrix in MineSet is computed prior to backfitting and is the same whether or not backfitting is applied. (See also “Backfitting”.)

---

Cost complexity is an advanced pruning method option developed in CART (Classification And Regression Trees). To find this pruning method, from the Tool Manager select the Classify Mining Tools tab and choose Decision Tree. Click on the Advanced Options button to get the dialog box showing Pruning Options. (You can also select the Regress Mining Tools tab and follow a similar path.)

Cost complexity pruning attempts to generate optimally sized trees by trading off the error rate of the tree (its cost) and the number of leaves in the tree (its complexity). During cost complexity, pruning the training set is partitioned into a learning set and a pruning set. The learning set is used to grow a pruning tree. This tree is pruned to generate a sequence of trees with decreasing complexity. The pruning set is then used to identify the minimum cost tree in this sequence. The size of the minimum cost tree is noted. The learning and pruning sets are recombined and used to grow a tree. This tree is then pruned to the size of the minimum cost tree.

The cost complexity pruning parameter allows you to select trees smaller than the minimum cost tree. The parameter indicates the number of standard errors more costly than the minimum cost tree that you are willing to accept. Setting the parameter to zero selects the minimum cost tree; setting the parameter to 0.5 selects the minimum size tree that had an error rate no more than 0.5 standard errors worse than the minimum cost tree. Higher numbers indicate more pruning. If your data might contain noise (errors and anomalies), increase the number to create smaller trees. If the tree is pruned back to a single node, decrease the number to decrease the amount of pruning and show more of the tree's structure.

Pruning is slower than limiting the tree height or increasing the split lower bound because a full tree is built and then pruned. Pruning, however, is done selectively, resulting in lower error rates.

Cross validation is a method of estimating classifier error that splits the dataset into a certain number (k) of folds (commonly 10), and builds that same number of classifiers. The process can be repeated multiple times to increase the reliability of the estimate. The inducer is trained and tested k times; each time it is trained on all the data minus a different fold, then tested on the holdout fold.

Cross Validation is an Estimate Error Option available from any inducer. The dialog box lets you set the number of folds in cross validation and the number of times to repeat the process.You can also set the random seed to a different number or maintain the same number to be sure you always cut the data from the same point.

Data may be in a difficult location, in an obscure form, or incompatible with other existing data. Fields may be missing or no longer valid. The process of bringing order from such complexity is called data cleaning, and many times is the precursor of data mining operations. Third party extraction tools are available to assist in this process.

To convert data from other formats to the MineSet format, and to do some rudimentary data cleaning operations, select File > Import Data from the Tool Manager menu bar. For more information, see the MineSet Enterprise Edition Interface Guide .

The Data Destination pane in the Tool Manager gives you these major choices: Viz Tools, Mining Tools, and Data File, all of which are located in the top level of tabs. Viz Tools provide a direct visualization of existing data to show correspondences that may provide insight. Mining Tools create models of data (and the accompanying visualization) that can then be used to predict future data. Data File is a storage option for saving manipulated data as a file.

Using the Data File tab in the Data Destination pane of Tool Manager lets you save manipulated data for future use in a data file on the client or server. If you click the Data File tab, a panel appears with two toggle buttons to let you specify whether the file is to be saved on the server or your client machine. The selected name for the client file appears next to the Client check box. If you select Client, the Choose new client file button brings up a dialog for you to choose the name for the client file. When you select Server, you can type the server filename directly into the adjacent text field.

---

Note: Pathnames are not permitted for server files; all server files are stored in the DataMove cache directory.

---

MineSet provides a data import facility, available from the Tool Manager File menu (Import Data). The Import Data panel allows you to choose the file you wish to import, along with a number of import options. See the MineSet Enterprise Edition Interface Guide for a description of these options, as well as a list of supported data formats.

The Data Transformations panel on the left of the main Tool Manager window lets you manipulate the tables of data in the dataset from which you will work. After you have selected a table using the File menu, the table's column headings appear in the Current Columns window of the Data Transformations panel. The functions of the displayed options are:

A decision table is a model that is hierarchical in structure like a decision tree, but breaks the data down at each level using a pair of attributes rather than a single attribute. The Decision Table inducer identifies the most important attributes (columns) for classifying the data, and then the visualizer displays them graphically as a nested table of cake charts. Each cake chart in the visualization can in turn be divided into smaller cakes representing the next most important attributes. Each visualization can contain several layers representing decreasingly important attributes. See the MineSet Enterprise Edition User's Guide for Windows for information on using the Decision Table Visualizer.

The method of classification used by a decision table is similar to that of a decision tree. Classification is done by picking the majority class of the region in which the example is found. If a record to be classified falls in a region where no training data occurred, classification is done by picking the predominant class one level up in the table hierarchy.

The Decision Tree is a predictive model. It makes predictions by using the dependent, or known, attribute values to help determine the value of the label, or unknown attribute. The task of predicting the value of a nominal value (usually character strings such as “yes” and “no”), or an attribute that can only take on a small number of values, is referred to as classification. A decision tree classifies data by predicting the label for each record. The underlying structure used for classification is a decision tree. Once the Decision Tree Inducer has built a classifier the data, the 3D Visualizer displays its structure.

The Decision Tree method shows how various attributes interact, that is, how combinations of attribute values affect the predicted label. The predicted label refers to the unknown characteristic in a given record. With the Decision Tree, how the data is distributed at subsequent nodes depends on the decision made at the previous node.

With the completed visualization, the bars at each node of the decision tree represent each label value. You can place the cursor over a bar to show the record count (or weight) and percentage for that label value. At the base of each node is shown the count (or weight) of the records that reach it.

The Discrete Label menu on the Classify tab of the Tool Manager provides a list of possible discrete labels. Discrete attributes (binned values, character string values, or a few integers) have a limited number of values. It is wise to select a label attribute with few values; for instance, two or three. If there are no discrete attributes, the menu shows No Discrete Label, and the Go button is disabled. You then must create a discrete attribute by binning or adding a new column using the Tool Manager's Data Transformations panel.

It is often useful to see the original data that corresponds to the selections in your current visualization. This is referred to as drill through. By selecting objects in a visualization, and sending the underlying data to the Tool Manager you can view the original data or analyze it using another tool. All tools in MineSet have this capability, although certain tools have additional characteristics which are detailed at the end of this entry.

If you choose Selections > Drill Through the Drill Through Dialog box opens. This dialog box gives you the following choices for viewing the selected records:

You can also choose one of the following

Only visualizations generated using the Tool Manager can be used for drill-through. Each .schema file for these visualizations includes a history section that informs the Tool Manager how that file was generated. A few special-purpose mining visualizations such as Learning Curves, Confusion Matrices, and Lift Curves, do not include a history and do not support drill-through.

When you select objects for drill through, you are implicitly specifying a filter statement based on the visualized table. If the table was transformed before visualization, the Tool Manager might have to change the filter to place it earlier in the history. For example, if the filter is based on a binned column, the Tool Manager must change it so it refers to the pre-binned column.

The filter cannot always be placed at the beginning of the history because the Tool Manager cannot adjust the filter for all operations. Specifically, if the filter refers to any column created via Add Column, Aggregate, or Apply Model, the filter cannot be placed earlier in the history than the operation that created the column. Furthermore, the Tool Manager can never move a filter earlier than an existing Sample operation, since that would dramatically change the output of the sampling.

The Show original data mode tries to truncate the history to show the records from as early a stage as possible. To prevent this mode from showing more records than were selected, however, the truncation does not remove any other existing filter operations (either user-created or the result of previous drill through).

Drilling down or drilling up refers to the act of clicking on an entity to see more or less detail. This is available only in the Map and Decision Table Visualizers.

Drill down using a right mouse click on an object. Drill up holding the Ctrl key and clicking the right mouse button on an object (or use the middle mouse button). Drilling up or down on the background performs the operation globally.

When a classifier is built, it is useful to know how well you can expect it to perform in the future (what is the classifier's error-rate). Factors affecting classification error-rate include:

The two common methods for estimating the error-rate of a classifier are described below. Both of these assume that future records will be sampled from the same distribution as the training set.

Generally, a holdout estimate should be used at the exploratory stage, as well as on datasets of over 5,000 records. Cross-validation should be used for the final classifier building phase, as well as on small datasets.

Backfitting applies the full dataset to a classifier or model that has been built using only data from the training set. When using holdout error estimation, you leave out a portion of the data for testing. When you backfit all the data through the classifier structure, you can reduce the error of the final classifier, because counts, weights and probabilities reflect the entire dataset. For a full discussion, see “Backfitting”.

The Evidence classifier model assigns each record in a dataset to a class. The Evidence Visualizer displays the structure of an evidence model. The visualizer can help you understand the importance of specific attribute values for classification. Also, it can be used to gain insight into how classification is done, as well as to answer “what if” questions. See the MineSet Enterprise Edition User's Guide for Windows  for information about how to use and interpret the Evidence Visualizer.

The Evidence Inducer builds a model that assumes the probabilities of each attribute value are independent given the class. For example, the iris dataset assumes that the four attributes (sepal length, sepal width, petal length, and petal width) are independent for each class of iris (iris-setosa, iris-versicolor, and iris-virginica). While this simplistic model is rarely true, the model is excellent for initial explorations of data and its classification prediction performance is very good in practical applications.

Each attribute value, or range of values (for binned continuous attributes) defines exactly one chart, which, in turn, gives the conditional probabilities for each class label. To classify a given record, one computes the probability of each class by multiplying its prior probability by the appropriate conditional probability from each row in the matrix. The final product gives the relative probability for each class and the highest value is the predicted class. If an attribute has an unknown value, it is ignored. These NULL values are represented by charts that are slightly offset from the rest of the charts and are on the left when you bring up the iris.schema file from the examples provided with MineSet. See Appendix A, “Sample Configuration and Data Files”, for pathnames, or follow the directions given in the MineSet Enterprise Edition User's Guide for Windows .

When you have the iris dataset shown with the Evidence inducer, the prior probability for each class label is depicted in the pie chart in the Label Probability Pane, on the right of the screen. The prior probability for a class label is the probability of seeing this label in the data for any randomly chosen record, ignoring all attribute values. Mathematically, this is the number of records with the class label divided by the total number of records. Toggle between Evidence (cakes) and Probability (pies) in the Main Window by clicking the Evidence label or by toggling the Evidence Mode in the View menu.

The conditional probabilities, depicted normally by cake charts in the Main Window on the left of the screen, show the relative probability of each attribute value given (conditioned on) each label value. The size of a cake slice indicates the amount of evidence the classifier adds to the prior probability after taking into account a given attribute value in a record. If the size of the slices are equal, the value is irrelevant, and the classifier adds the same amount of evidence to all classes.

Technically, the slice of the chart represents the normalized conditional probability of an attribute value A, given the class label L. The conditional probability, P(A|L), is the probability that a random record chosen only from records with label L takes the value A. Under the default settings, the probability is computed based on record weights. For example, P(0.75 < petal width < 1.65 | iris-versicolor) is 91.6, because there are 36 records with label iris-versicolor, and 33 of them have a petal width in this range.

The height of the gray base in bars/evidence mode shows the evidence for the selected label. This overall evidence value is computed by -log[ 1-P(L)/sum(P(Li))] for evidence for, and by -log[ P(L)/sum(P(Li))] for evidence against. The subtract min evidence toggle affects the base the same as the small bars.

Importance is a measure of predictive power with respect to a label. The Main Window provides valuable insight not only into the importance of each attribute value affecting the class value, but also into the importance of specific attribute values.

*minesetQuiet:TRUE 

The File Menu for most of the visualizers is similar and can contain a number of options:

Most MineSet visual tools require at least two files in addition to a configuration file: the .data file and the .schema file, which are created automatically when you access the tools with the Tool Manager.

The .data file consists of rows of tab-separated fields. You can also generate data files by extracting data from a source (such as a an Oracle, INFORMIX, or Sybase database) and formatting it using your favorite text editor (such as WordPad, jot, vi, or Emacs). See the MineSet Enterprise Edition Interface Guide for the required file format. Table 1-10 gives the types of file extensions created for each visual tool when you use the Tool Manager.

When starting a visualizer or when opening a file, you must specify the configuration file, not the data or schema file. The Record Viewer, however, opens any .schema files.

Data files can have user-defined extensions (the sample files provided have a .data extension).

This button on the Data Transformation pane of Tool Manager lets you filter the data via a mathematical expression. The resulting table includes only records for which the expression is true (or, if numerical, non-zero). When you click Filter, the Filter dialog box appears.

This dialog box lets you select column names and operators on the left to build an expression on the right. For a complete description of the expression definition language, see “Adding Columns”.

The Filter Panel is accessible from the View pulldown menu. This section describes the options. The Scatter, Splat, Map, and Decision Table Visualizers use this panel. The Tree Visualizer has a similar panel. The Evidence Visualizer has its own variant.

Filter Panel brings up a panel which lets you reduce the number of entities displayed in the main viewing area, based on one or more criteria. You can use the filter panel to fine-tune the display, emphasize specific information, or simply shrink the amount of information displayed.

Scale to Filter (only in the Scatter Visualizer) lets you specify whether the heights of the graphical objects are scaled across the entire dataset or just across the filtered data.

The filter panel has two sections. The top part lets you filter based on string variables. To select all values of a variable, click Set All. To clear the current selections, click Clear. To select a value, click it. To deselect a value, simply click it again.

The bottom part lets you filter based on the values of both string and numeric variables. Only variables whose values do not change as you navigate the slider can be used in filtering. Scrolling the pane reveals the hierarchy (Contains, Equals, Matches, IsNull).

To filter numeric values, enter the value and select a relational operation (=, !=, >, <, >=, <=). To filter alphanumeric values, enter the string. You can use any of three types of string comparisons:

For example, California matches Cal*, Cal?fornia, and Cal[a-z]fornia.

In some cases (usually associated with binning in the Tool Manager), an option menu of values appears, instead of a text field. To ignore that variable, select Ignored in the Option menu. You can use relational operators (such as >=) with these options. This means that the specified value as well as subsequent ones are selected.

In addition to numeric and string comparison operations, you can specify Is Null, which is true if the value is null.

To the right of each field is an additional option menu that lets you specify “And” or “Or” options. For example, you could specify “sales > 20 And < 40.” You can have any number of And or Or clauses for a given variable, but cannot mix And and Or in a single variable.

Click the Apply button to start filtering. If you press Enter while the panel is active, filtering starts automatically.

Gain Ratio is a Decision Tree splitting criteria consisting of the Mutual Info divided by the entropy (change in purity) of the split made while ignoring the label values. Refer to “Decision Tree” for a discussion of the Decision Tree splitting criteria.

Besides the F1 or Shift-F1 keyboard key combination, the Help menu in each tool provides access to five help functions:

Besides the F1 key, the Help menu in each tool provides access to all of the MineSet Enterprise Edition manuals. Product Information brings up a screen with the version number and copyright notice for the visualizer.

The Histogram Visualizer automatically bins all of the continuous-type columns in the data and sends the result to the Statistics Visualizer. See “Statistics Visualizer ”. You can set the following Histogram Visualizer options:

You can get the history of previous operations in the current session by using the Table History buttons in Tool Manager. See “Table History Buttons”. You can also save your work using the File pulldown menu. You can also save the data file after completing your transformations by clicking the Data File tab in the Tool Manager Data Destinations pane and then saving the file. When you resume work, load the appropriate session or file you want to work with and the history is there.

Holdout refers to the process of training a model on a sample of the dataset so that the model may be tested on the remaining portion. This procedure assesses the model's error rate based on data which was not used to train the model. The fraction of data which is used for training is referred to in MineSet as the Holdout Ratio.

An inducer is an algorithm that builds a predictive model from a training set, which consists of records with labels. The training set is part of the dataset used by the inducer to “learn” how to construct the model. Once the model is built, its structure can be visualized or used to classify unlabeled records. Running inducers can be a CPU- and I/O-intensive process. For this reason, the MineSet inducers run on the MineSet server, rather than on the MineSet client (see Figure 1-19).

Figure 1-19. Tool Execution Sequence for Models

Inducers require a training set, which is a table with attributes or characteristics, one of which is designated as the label. Once a model is built, it can predict this label for new records. These new records must be in a table that has all the attributes used by the model, with the same name and type as they were in the training set. The table need not contain the label attribute. If it exists, it is ignored during prediction.

There are four modes for running an inducer.

The Classifier and Error mode (or Regressor and Error mode) uses a holdout method to build a model: a random portion of the data is used for training (commonly two-thirds) and the rest for testing. This holdout proportion can be set in Advanced Options on Windows systems, or Further Inducer Options on IRIX systems (see “Error Estimation”). This method is the default mode and is recommended for initial explorations. It is fast and provides an error estimate.

The Classifier Only (or Regressor Only) mode uses all the data to build the model. There is no error estimation. Use this mode when there is little data or when you build the final model.

The Estimate Error mode assesses the error of a model that would be built if all the data were used (as with Classifier Only or Regressor Only mode). Estimate Error uses cross-validation, resulting in long running times. (See “Cross Validation”.) Use this method when there is little data. The induced model is exactly the same as the one induced by the Classifier Only or Regressor Only mode.

The Learning Curve mode assesses the effect of training set size on the error rate of a classifier (see “Learning Curve”).

After you press Go! in the Data Destination panel, the Status Window at the bottom of the Tool Manager's main window shows the algorithm's progress and displays specific information for the induced model. For example, for Decision Trees it shows the number of nodes, the number of leaves, and the depth of the Decision Tree. This information is saved automatically on your workstation under the session file name with a .out extension.

On Windows systems, setting the locale is done through the desktop Control Panel element “Regional Settings.” This section deals with setting the locale on IRIX systems.

MineSet supports international datasets. Text labels in the graphical interface appear in English, but you can view multibyte column names and data values in the language corresponding to the data encoding. MineSet automatically supports EUC encoding for Japanese, Chinese, and Korean, provided the corresponding language product is installed. For other languages and encodings, see “Extending to Other Languages and Encodings (IRIX Only)”.

locale -a 

setenv LANG ja_JP.EUC 

Iterative k-means is a method used by MineSet's clustering tool. This method will automatically choose the number of clusters within a user-selected range; no specific number of clusters is needed. The algorithm chooses the number of clusters by generating several candidate clusterings and choosing the best one using a combination of the dispersion metric and a user-selectable choice point.

The Evidence Visualizer uses an induction process whereby counts (or weights) are used to calculate probabilities. Laplace correction avoids extremes of probabilities (zeroes and ones).

To find the Laplace correction function, from the Tool Manager Mining Tools click the Classify tab and choose Inducer: > Evidence. When you click the Advance Options button the resulting dialog box shows the Laplace check box under “Evidence Options.”

The Laplace correction may be used in an example in which we may prefer not to assign a a probability of 1 to the event “a patient tested positive for AIDS has a deadly disease.” Assigning a probability close to 1 (but not 1), allows for errors or unrepresentative samples. Laplace correction biases the Evidence Visualizer's probabilities towards the average, thus avoiding the extremes of 0 and 1.

This means every chart in the Evidence Main Window has a non-zero slice for each class. The fewer the records in a bin, the more it is changed towards the average. If the Laplace correction is checked, and the factor is left empty or set to 0, an automatic Laplace correction is applied, using a heuristic that applies a factor of 1/training-set-weight.

A learning curve is a graph that shows the error of the model generated by an inducer as a function of the number of records used to create the model. Typically, the more records used to generate the mode, the lower its error.

The learning curve is created by generating the specified number of models for each of the points on the curve. Each model is generated using a random sample of the records, and its error is estimated using the rest of the records (those not used for training).

Generating a learning curve takes a significant amount of CPU time. If t_i is the time to train an inducer on training set i (where i ranges from 1 to the number of points), and there are k runs per point, the total time is

Increasing the number of runs per point increases the running time proportionally, but improves the estimate of the average. The default value of the number of runs is 3.

The Scatter Visualizer's filter panel can be used to filter some of the data types shown (average points, confidence intervals, interpolated points, or actual trials). For example, you might want to remove the data points for the trials and confidence intervals and show only the averages and interpolated points.

Figure 1-20. Learning Curve

Learning Curve is a mode in the Classify menu (or the Regressor menu) of the Mining Tools tab. It can be used with any of the inducers. When the Learning Curve mode is selected, the Advanced Options dialog box (or Further Options dialog box) lets you specify Learning Curve Options on the right of the dialog box, including:

The number of records to use at each intermediate point is calculated automatically.

The number of points in the learning curve must be specified; also, it must be greater than or equal to 1. The number of records for the starting and ending points can be specified to allow generating a learning curve for a specific range of the training set. If either of these options are left blank, they are calculated automatically based on the number of points in the learning curve and the total number of records in the training set. This default covers the entire range of the training set. For instance, assume a file containing 80,000 records. If you specified 3 points in the learning curve, the algorithm generates points at 20,000, 40,000 and 60,000 records. Often it is useful to “zoom in” on a smaller range. For example, a learning curve might be generated only for a range of 1000 to 10,000 records.

A lift curve graphs the difference between a random ordering of records, and the order created by the classifier when describing a particular label value. For example, you created a model to predict which customers are likely to churn, and now want to target those customers before they churn. The lift curve helps accomplish this goal.

A lift curve is a plot in which the X axis shows the number of records from 0 to 100% and the Y axis shows the number of records corresponding to customers who have a given label value (Churn=yes in our case). Two curves are shown on the graph Figure 1-21.

The lower curve (red) shows the number of customers expected to churn given a random ordering of the records. The upper curve (white) shows the percentage of customers who churn when placed in order according to the classifier's score (probability estimate) for each record. Records representing customers that the classifier identifies as most likely to churn appear first; those less likely to churn appear last. The advantage that the classifier ordering provides can be seen by the difference between the classifier curve and the random curve.

Figure 1-21. Lift Curve

In building this lift curve, a selected classifier is applied to the test set. A specified segment of the dataset is used for training, then the induced classifier run on the remainder of the dataset. The Display Lift Curve option is a check box under Advanced Options (or Further Options) for all inducers when using Classifier & Error mode. A Lift Curve requires a label value to be chosen. A lift curve is generated and displayed for that label value.

The purpose of a loss matrix is to control which types of errors the classifier will make. In many situations, some errors are costlier than others. A good example comes from the mushroom dataset, in which mushrooms are classified as edible or poisonous. Misclassifying an edible mushroom as poisonous can cost a few dollars (the price of the uneaten mushroom). Misclassifying a poisonous mushroom as edible can cost thousands (the price of a hospital stay). Classifiers use a loss matrix to avoid such costly errors.

The loss matrix is used most effectively in conjunction with a confusion matrix. The confusion matrix identifies the types and quantities of errors made by the classifier. If the confusion matrix indicates that the classifier is making a large number of costly errors, giving costly errors a high weight in the loss matrix may be the best way to improve the classifier's performance. The Use Loss Matrix check box under Advanced Options (Further Options on IRIX) activates the loss matrix. It is available for all classification inducers.

The process is demonstrated in the following example: Figure 1-22 shows a confusion matrix for the mushroom dataset using the Decision Tree Inducer when only a ratio of 0.1 (10%) was used for a training set.

Figure 1-22. Confusion Matrix for the Mushroom Dataset Using Defaults Settings

Eight records, representing poisonous mushrooms, were classified as edible (0.1%); 15 records, representing edible mushrooms, were classified as poisonous (0.2%); 3793 edible mushrooms and 3496 poisonous mushrooms were correctly classified. While the error-rate for the classifier is only 0.31% (less than one percent), our estimated loss would be $10000 * 8 + $2 * 15 = $80,030 or worse.

Figure 1-23 shows a confusion matrix for the same dataset, but with the Decision Tree Inducer run using a loss matrix representing the above costs. The new classifier is very conservative and makes no mistakes in classifying a poisonous mushroom as edible; but it makes 1558 mistakes (1543+8) in classifying edible mushrooms as poisonous. The total estimated loss we would incur is thus $10000 * 0 + $2 * 1558 = $3116, only 3% of the cost of the classifier that did not take losses into account.

Figure 1-23. Confusion Matrix for the Mushroom Dataset With Loss Matrix

Loss matrices also allow predicting unknown (null values), which are shown as question marks (?). For example, suppose it costs us $1 to ask an outside expert whether a mushroom is poisonous or edible. In that case, some classifications result in an unknown prediction. Running the Decision Tree Inducer yields the confusion matrix shown in Figure 1-24 where there are 1551 unknowns, and only 15 edible mushrooms are classified as poisonous. The overall cost is thus $10000*0 + $1 * 1551 + $2 * 15 = $1581.

Figure 1-24. Confusion Matrix for the Mushroom Dataset With Loss Matrix Allowing Unknown Predictions

Note that for decision trees, loss matrices are based on probability estimates made at the leaves of the tree. For reliable estimates:

The Use Loss Matrix option is a check box under Advanced Options (or Further Options) for all classification inducers. The Edit matrix button can then be used to define the loss matrix. To avoid unknowns from being predicted, fill the unknown prediction column with the highest value in the matrix.

If a Loss Matrix has been specified in the Evidence Visualizer's Advanced Options (or Further Inducer Options on IRIX systems), a button labelled Use Loss Matrix appears to the lower right of the probability pie. When selected (the default) the Loss Matrix is used to adjust the probabilities shown. The largest slice shown is the class that takes into account the Loss Matrix. To see what the probabilities would be without using the Loss Matrix, deselect the Use Loss Matrix button. When using the Loss Matrix, a gray slice may be present because there was a column for predicting null when you edited it. If the gray slice is the largest slice, the classifier predicts null.

The Map Visualizer shows you data displayed over a three-dimensional landscape of bar chart shapes. It is useful when data has a geographical context, although it can be useful in creating any spatially-related visualization.

Data items are associated with graphical bar chart objects in the visual landscape. However, the objects have recognizable shapes and positions. The landscape can consist of a collection of these objects, each with individual heights and colors (see Figure 1-25). Besides dynamically navigating through this landscape, you can drill up and down to aggregate or see increased granularity, as well as use animation to see how the data changes across one or two independent dimensions.

Figure 1-25. Sample Map Visualizer Screen Showing 1990 U.S. Population

The landscape can also consist of a flat plane of these geographical objects drawn as simple outlines, with “bar chart” cylinders placed at specific locations.

Another landscape possibility is lines with endpoints at specific point locations, all with individual widths and colors. Lines have width and color properties, instead of the height and color properties of the arbitrarily shaped objects and cylinders.

*minesetQuiet:TRUE

From the Data Destination pane of Tool Manager, clicking the Mining Tools tab exposes the following tabs:

In most of the tools, selection is done using Shift-left mouse click. Clicking the left mouse button on an object without pressing Shift, selects the object under the cursor while deselecting all other previously selected objects. Holding down Shift while clicking the left mouse button toggles the selection of that object without affecting any other selections. (The Splat Visualizer has a different interface, described in the “Splat Visualizer” entry.)

When you select an item, a message describing that item appears in the tool's main window; by default, the visual tools only show information on the last object selected. A separate Record Viewer window displays a table, which shows the values for all selections. In the Tree and Map Visualizers, choose the Selection > Show Values to see this.

If a message has been set for the particular tool, that message also appears in the table. Columns in this table can be resized by dragging the separators between the columns. You also can click on a value to display the complete text of that value at the top of the table.

Mutual Info is a Decision Tree splitting criterion that governs the way the classifier makes its decisions. Mutual Info is the change in purity (that is, the entropy) between the parent node and the weighted average of the purities of the child nodes. The weighted average is based on the number of records at each child node. See “Decision Tree” for discussion of other splitting criteria.

The Evidence Inducer is sometimes called Naive-Bayes or Simple Bayes. It builds a model that assumes the probabilities of each attribute value are independent given the class. For example, for the iris dataset that the four attributes (sepal length, sepal width, petal length, and petal width) are independent for each class of iris (iris setosa, iris versicolor, and iris virginica). While this simplistic assumption is rarely true, the model is excellent for initial explorations of data and its classification prediction performance is very good in practical applications.

This section consists of three tables that serve as a quick reference for the Evidence, Decision Table, Map, Scatter, and Splat Visualizer navigation controls. Table 1-12 describes the navigation buttons.

Table 1-12. Navigation Buttons in Non-Tree Visualizers

Button	Name	Action
	Pick	Changes the program to pick mode (an arrow). In pick mode, you can highlight (brush over) or select (click) elements of the chart.
	Grasp	Changes the program to grasp mode (a hand). In grasp mode, you can move the chart around in the window: — To rotate chart, hold down left mouse button and move mouse. — To move chart in window, hold down left and right mouse buttons (or use middle mouse button if your system is configured for a three-button mouse) and move mouse.
	Home	Returns the chart to the size and position designated as the home view. By default this is the size and position of the chart when the visualizer is first invoked. You can change the home position by using the set home icon.
	Set home	Sets a new home view for the chart. Use this when you want to save a certain view or position.
	View All	Moves the chart to a position where it is centered and all of it is visible in the window.
	Zoom	Moves the point you select to the middle of the pane and zooms to it. When the mouse cursor becomes a targeting sight, move it to the spot you want to see more clearly, then press and hold the left and right mouse buttons (or use the middle mouse button if your system is configured for a three-button mouse).
	3D	Toggles the 3D perspective.
	Top View	Changes the chart to a top view (Scatter and Splat Visualizers only).
	Front View	Changes the chart to a front view (Scatter and Splat Visualizers only).
	Side View	Changes the chart to a side view (Scatter and Splat Visualizers only).

Table 1-13 describes the adjustment sliders and wheels in the non-tree visualizers.

Table 1-14 lists several manipulations you can to perform on the charts in the non-tree visualizers.

The Tree Visualizer display is best thought of as though you are viewing the scene through a camera. To change the view, you change the position of the camera (the viewpoint). This section consists of two tables that serve as a quick reference for the Tree, Decision Tree, Option Tree, and Regression Tree Visualizer controls. Table 1-15 describes the navigation buttons.

Table 1-15. Navigation Icons in the Tree Visualizers

icon	Action
	Returns the chart to the size and position designated as the home view. By default this is the size and position of the chart when the visualizer is first invoked. You can change the home position by using the next icon.
	Sets a new home view for the chart. Use this to save a certain view or position.
	Moves the chart to a position where it is centered and all of it is visible in the window.
	Undoes the previous move (like the Back button on a Web browser).
	Redoes a move that has been undone (like the Forward button on a Web browser).
	Moves one node closer to root of tree.
	Moves one node or bar to left.
	Moves one node or bar to right.
	Moves one node down the tree on the left path.
	Moves one node down the tree on the right path.
	Pops up a menu of possible paths from the current node.

Table 1-16 describes the adjustment sliders and wheels in the tree visualizers.

The Nominal Order menu on the Evidence Visualizer and Decision Table lets you control how values for nominal (named) attributes are ordered, and offers these choices:

The Decision Tree tool uses Normalized Mutual Info as the default splitting criteria. Technically, this is the Mutual Info divided by the log (base 2) of the number of child nodes. Refer to “Decision Tree” for a discussion of the Decision Tree splitting criteria.

Some visualizers, such as Splat Visualizer and Map Visualizer use special representations when fields with unknown data values, or nulls, are mapped to visual attributes. (For a discussion of null values, see the MineSet Enterprise Edition Interface Guide chapter “Nulls in MineSet.”) When every record in a bin has a null value for the column mapped to color, the resulting color for that splat is gray. If one or more records in the aggregate have non-null values for the column mapped to colors, then that value is (or those values are) used to compute the color. While the sum of a value and null is null, the average of a value and null is the value (that is, value + Null = Null; avg(val, Null) = val).

When a null value is displayed in the various visualizer display window areas: Pick Window, Selection Window or “Pointer is Over” area, it is shown as a question mark (?).

For numeric columns containing nulls which are mapped to axes, there is a special null position below the range defined by the axis. This is to help show that the null value is discontinuous with the other values. The null positions for numeric axes can be turned off using the Show Null Positions option under the View Menu. For string-valued columns mapped to axes, nulls (represented by a `?') are treated as just another value.

In the Map Visualizer, nulls can occur when any of the following is true:

The Option Tree is a predictive model. It makes predictions by using the dependent, or known, attribute values to help determine the value of the label, or unknown attribute. An Option Tree starts classifying data by assigning each record to a class. The underlying structure used for classification is a Decision Tree, as described in “Decision Tree”. Option Trees extend a regular Decision Tree model with Option Nodes. An Option Node shows several options that can be chosen at a decision node in the tree. Option Trees tend to be much larger and more complex than ordinary decision trees.

Parallelization is offered through the multiprocessor version of MineSet for IRIX systems. It allows compute-intensive tasks to be performed in parallel. To run the 64-bit version, install the parallel server and choose parallel version from the Tool Manager Preferences menu.

DataMove is a process that runs on the server, and provides access to databases and data stored in flat files, and transforms data for the mining and visualization tools.

Large memory (64-bit) is supported on IRIX 6.4 and later releases. If you have IRIX 6.2, you can still use the 32-bit data mining utility, but you must upgrade to IRIX 6.5 in order to obtain 64-bit support and p-threads. To get the full advantage of 64-bit addressing you may also need to change the systune resource parameters, depending on your system configuration.

The systune parameters determine the default limits on the available system resources. Table 1-17 lists the systune parameter values recommended on IRIX (for more details see the systune(1M) reference page).

---

Note: You must reboot your machine after setting the new parameters.

---

If you have installed the multiprocessor version of MineSet, it is possible to compute tree-based algorithms in parallel whenever a branch contains over 1000 records. Each node on the tree estimates the best possible split on the corresponding level, and these tasks are performed in parallel. The maximum number of threads a program can spawn is determined automatically by default. You can control the number of threads by changing the parallelization mode in the Preferences panel of Tool Manager. Parallelization may cause memory fragmentation, causing the largest data sets that can be computed in parallel to be smaller than the largest data sets that can be computed on a single processor.

When working with Association Rules, if you visualize a grid with right-hand side (RHS) and left-hand side (LHS) as two axes, predictability describes the fraction of items occurring in the RHS out of all items with LHS, or the prevalence divided by the frequency of occurrence of LHS items (see “Association Rules Visualization” for an explanation of RHS and LHS). It indicates how often X and Y occur together, stated as a fraction of the number of records in which X occurs. For example, if the predictability is 50%, X and Y occur together in 50% of the records in which X occurs. Therefore, if you know that X occurs in a record, you can expect that 50% of the time Y occurs in that record. For example, if the record shows a 50% predictability in the correlation between men buying baby diapers and cigarettes, then 50% of men buying baby diapers will also buy cigarettes.

When working with Association Rules, if you visualize a grid with right-hand side (RHS) and left-hand side (LHS) as two axes, prevalence describes the frequency of items occurring in LHS and RHS together (see “Association Rules Visualization” for an explanation of RHS and LHS). Prevalence quantifies how often X and Y occur together in the file as a fraction of the total number of records. For example, if the prevalence in 1%, X and Y occur together is 1% of the total number of records.

Pruning is a Decision Tree inducer option that may be changed to affect tree height and Split Lower Bounds. The default pruning factor is 0.7. Higher numbers indicate more pruning. See “Decision Tree” for a full discussion.

A random seed may be specified in several option dialog boxes in MineSet. It refers to the starting point of a random number generating algorithm. The random number generated is then used in the sampling of the dataset. By using the same random seed each time you take a sampling, for example, you can be sure of working with the same selection of data each time you refine your model. If you change the random seed, you get a different selection from the basic dataset.

The Record Viewer allows you to view your data directly. This gives you the opportunity to get familiar with the columns and the data values within them. The Record Viewer also allows you to:

In certain experimental designs, portions of the true population are sampled more frequently than others. For example, while you might want a 1% sample of some population, a small minority that is already 0.1% of the population results in a 0.001% sample, which might be too small (for instance, you might get two people). Record weighting lets you give each record a weight; thus, a subpopulation that was sampled twice as frequently might get a weight of 0.5, while the rest of the population is given a weight of 1.

As another example, a phone company stores all fraudulent phone calls in the dataset, while storing only a small fraction of non-fraudulent calls. By using record weighting, it is possible give each record its true portion of the population.

Finally, some datasets are already aggregated, and the records have a natural “count” associated with them (for example, statistics about cities in the U.S. usually have an associated count of the population). This count attribute can be mapped to weight, which is equivalent to replicating each record by the number of counts.

The semantics of record weighting is that a record weight of 2 is equivalent to two records with a record weight of 1. Floating point weights are allowed. See also “Weighting”.

The Regress tab allows you to access MineSet's regressor. From the Data Destinations pane of Tool Manager, click the Mining Tools tab to reveal the Regress tab.

A regressor is a predictive model that performs regression. Regression is the task of predicting a continuous label value, given a set of descriptive attributes. Regression and classification are similar, the difference being that in classification the predicted label can take on only a small number of discrete values.

When a regressor is generated, MineSet also generates a visualization. This visualization can help you understand the regressor and how it makes predictions. In addition, it can provide valuable insight into the data itself. Once generated, a regressor can be used to predict the label value for unlabeled records.

The generated regressor is named with the prefix of the session filename (as determined in Tool Manager) and the suffix -rt.regress, for example churn-rt.regress. By default, all regressors are stored on the server in the file_cache directory, which defaults to MineSet Files on Windows and mineset_files on IRIX. Windows systems store the visualizations in the current working directory as a 3D bar chart icon. The files have a -rt.treeviz suffix. 

These regressors can be used to predict the labels for unlabeled datasets. To apply a stored regressor, you need to select a dataset of records with the attributes used by the regressor. The regressor will predict a new continuous label value for each record. See “Apply Model” and “Backfitting”.

When you need to simplify your visualization or model, you can remove columns from your current work without removing them from the dataset. From the Data Transformations pane of the Tool Manager, select a column from the Current columns text pane and click the Remove Column button. This removes the column from your calculations without removing it from the dataset.

A Return-on-Investment (ROI) curve is similar to a Lift Curve, but displays accuracy in terms of loss rather than in terms of error; taking into account the Loss Matrix used.

The ROI curve configurable with the Tool Manager from any inducer's Advanced Options panel, as a check box under the Error Estimation Options section. It is not available for Regressors.

The points in an ROI curve are ordered by the expected loss for each record, were they to be labeled by the chosen label value. Similarly, the height of each point in the curve indicates the cumulative profit (inverse loss), rather than the cumulative accuracy (inverse error) of all records up to this point.

The expected loss is computed by multiplying entries in the Loss Matrix, under the chosen labels column (see “Loss Matrix”) by the probabilities assigned to the corresponding classes, for the classes by the classifier. Hence, if the classifier is very sure about its prediction, the expected loss will be low, and the record will appear near the left side of the ROI curve.

Figure 1-27. Return on Investment Curve

The idea behind the ROI curve is that the user will take an action for each individual record in the dataset. That action will be the one associated with the chosen label value. For example, in the churn dataset, the action associated with the label Yes, might be to send that person some marketing material. This might stop the person from churning; but the action is costly if done indiscriminately. The peak of the ROI curve shows approximately how much money would have been saved on the test set, if the classifier was used to predict whether or not to send the mailing to a particular person.

Special care needs to be taken when filling out a Loss Matrix for use with an ROI Curve. The column under a certain predicted label determines the resulting ROI curve for that label value. The entries in this column need to represent the expected gain or loss for taking the action associated with that label value, on all of the possible classes. For example, the entry under the column “prediction yes” in churn, under the row “actual value no”, may contain the value 2 to indicate that the cost of mailing a brochure (the action associated with “yes”) to someone who was not going to churn, is $2. On the other hand, the entry under the column yes, row yes, may have a value of -10 to indicate that a customer was prevented from churning, saving the company $10 over the cost of the mailing.

MineSet provides a range of sample files for users to work with. These are covered in detail in the previous section. Directories vary according to platform.

Windows users find the example files in the directory in which MineSet was installed, under \examples. 

UNIX users find the files in/usr/lib/MineSet/examples. 

You can save the session you are working on using the Tool Manager's File pulldown menu, or save particular transformations by clicking on the Data File tab in the Tool Manager's Data Destination pane. When you return to work, load the session or file from Tool Manager's File pulldown menu and the file and history resumes from there.

The Scatter Visualizer lets you visually analyze relationships among several variables, either statically or by animation. It is particularly useful for seeing individual data points when you do not have a large number of records (less than 50,000), or if you have aggregated a large number of records into a small set of distinct aggregates. If your dataset has a very large number of records consider using the Splat Visualizer. Analysis in the Scatter Visualizer is done using:

The Scatter Visualizer lets you visualize your data by mapping each record, or row, in the dataset to an entity in the three-dimensional landscape. Variables in the data can be mapped to the sizes, colors, and positions of the entities. Also, you can map one or two numeric variables to the sliders in the animation control panel. If the variables mapped to sizes, colors, or positions of the entities depend on the variables mapped to sliders, the sliders can be used to drive an animation. For example, the data might represent the sales of several companies over time. If the time variable is mapped to a slider and the sales variable is mapped to size, then the entities grow or shrink as the time slider is animated.

After you create a visualization of your data, the Scatter Visualizer lets you analyze the data in various ways. The animation control panel lets you trace animation paths in one or two dimensions. By playing back the path you created, you can watch the size, color, and motion of the entities for trends or anomalies. In the three-dimensional landscape, you can orient the display to emphasize particular dimensions or a point of view. The Scatter Visualizer lets you scale the values of variables to give them greater emphasis. Also, you can filter the display to show only those entities meeting certain criteria.

The Selection menu is similar for all tools; functions not available for a particular tool do not appear on the menu. The Selection menu lets you drill through to the underlying data. To perform a drill-through, first select one or more entities or splats, then chose one of the two methods of drilling through to the underlying records.

For further details on drilling through, see “Drill Through”.

You can map a column to a slider to show how a value changes according to a specific criteria. Columns can be mapped to sliders if that column is numeric (int, float, double) or binned. If the column is already binned it will have _bin after the name. Column type is noted after the name of the column in the Current Columns field, for example total day calls - double.

You can create sliders for the Map, Scatter, and Splat Visualizers automatically by mapping any numeric column to either or both sliders from the Current Columns in the Data Transformations pane of Tool Manager. See “Animation Buttons and Sliders”.

You can sort column names in alphabetical order for ease of reference, without changing the dataset. From the Data Transformations pane of the Tool Manager, click the Show columns sorted check box (Windows), or the Sort Column Names button (IRIX).

The Splat Visualizer lets you visually analyze relationships among several variables, either statically or by animation. It is particularly appropriate for datasets with large numbers of records. Choose the Scatter Visualizer if you want to see individual data points and do not have a large number of records. Data analysis is done using

The Splat Visualizer lets you visualize your data by mapping columns to axes, sliders, color, and opacity. The resulting three-dimensional landscape can be thought of as an approximation to a scatterplot in which every datapoint is drawn separately. It is not truly a scatterplot, because datapoints that are close together (fall in the same bin) are aggregated and drawn as a single splat.

Each numeric column that is mapped to an axis or slider first must be binned. If this binning step is skipped, the Tool Manager does it using automatic uniform binning (see “Binning”). String columns can be mapped directly to axes. Any numeric column can be mapped to a color. The color of a splat is derived by averaging the value of the column mapped to color for all the data points that fall in a bin. The opacity of a splat is based on a weighting of the number of datapoints that fall in a bin. If nothing is mapped to opacity, record counts are used to determine it. The interactivity of the resulting visualization is independent of the number of data points represented; it depends only on the number of bins in the axis dimensions.

If your dataset is very large, aggregate explicitly in the Tool Manager. This causes the server to perform the processing, rather than having the entire dataset sent to the client and aggregated there. See “Aggregate”.

Up to two numeric columns can be mapped to the sliders in the animation control panel. The splats change their color and opacity during animation as the sliders in the animation panel are moved from point to point along the slider `s path. Unlike the Scatter Visualizer, neither the position nor the size of the splats change; they are at fixed, uniformly spaced positions. Only their color and opacity change, which can give the illusion of actual movement.

If a string column is mapped onto an axis, binning is defined to be the distinct values of that column. The order of the values along a string axis is automatically determined by sorting the distinct values by the average aggregate value of the column mapped to color. Looking at the color changes along a string-valued axis lets you see how well that column correlates with the column mapped to color. If no color is present, then the opacity variable is used to determine the order.

Splat Visualizer Opacity

The column mapped to opacity should be record count or a column used to weight records. A splat's opacity, a, is based on this column according to the following relation:

where weight is the column mapped to opacity (or the record count if no such column was mapped to opacity). The shape of this function is such that the opacity asymptotically approaches 1 (totally opaque) as the value of weight becomes large. The variable u is what is scaled when you adjust the opacity scale slider on the left of the main window. Figure 1-29 shows the shape of this function for low and high values of u. Figure 1-30 shows the same visualization with low and high values of u.

Figure 1-29. Shape of Opacity Function For Low and High Values of u 

Figure 1-30. Image Where u = 5.3, and u = 30

If nothing is mapped to opacity, the Splat Visualizer generates a column of ones to produce record counts when aggregating. This means all records are weighted equally. A sum aggregation is done on this column, and an average aggregation is done on the column mapped to color while grouping by all the axis and slider columns. All other columns are unnecessary and removed. You do not need to map anything to opacity unless you want each record to be weighted by something other than 1.

See the MineSet Enterprise Edition User's Guide for Windows

 for details about processing in the Tool Manager rather than the client.

When you invoke the tool, all the processing is done on the server, and that the datafile, adult94.splatviz.data, contains rows that are aggregates of rows in the original data.

In some cases, you might have a column by which you want to weight the records. For example, if you have a dataset for which one column was population and another was average salary (which you want to map to color), you can map population to opacity, and average salary to color; then have the Splat Visualizer do the aggregation. This aggregation groups-by the axis and slider columns, so that it sum aggregates the opacity column (which, in this case, is population). The new column is called sum_population. The average_salary column is revised, so that it is still average salary, but weighted by each row's population. In this way, the average salary column still shows the average salary for all the people it represents.

Alternatively, if you want to avoid client-side processing and storage because of the size of your dataset, you can perform the same aggregation in Tool Manager by doing the following:

1. Create a new column, defining temp = population * avg_income.

2. Perform an aggregation: group-by axis and slider columns, sum aggregate population, and sum aggregate temp.

3. create a new column, defining
   avg_salary = sum_temp/sum_population
   This creates the weighted average.

4. Now you can map sum_population to opacity, and avg_salary to color.

Note that these steps are the ones automatically taken by the Splat Visualizer if you do not explicitly do them in the Tool Manager. However it is more efficient if you perform them in Tool Manager to that the work is performed on the server, resulting in a considerably smaller file being retrieved to the client.

*minesetQuiet:TRUE

Split Lower Bound is an option for refining the Decision Tree Inducer and Regression Tree Inducer. Increasing it tends to produce smaller trees, but may impair accuracy.

Split Lower Bound is a lower boundary on the weight (normally the number of records if weight was not set) that must be present in at least two of the node's children. The default for this option is 2. For example, if there is a three-way split in the node, at least two out of the three children must have a weight of two or more (two records or more if weight is not set). This provides another method of limiting the size of the Decision Tree.

Increasing the split lower bound tends to increase the reliability of the probability estimates, because the number of records at each leaf is larger. It also creates smaller trees and decreases the induction time. If you expect the data to contain noise (errors or anomalies), or if you use the tree for estimating probabilities (see “Apply Model”), increase the split lower bound to 5 or more. If your dataset is very small (< 100 records), you might want to decrease this number to 1. See “Decision Tree”.

This option offers three splitting criteria selections in Decision Tree. The definitions below are technical. For a given problem, it is difficult to know which criteria will be best. Try them all, and select the one that leads to the lowest error estimate, or to a Decision Tree you find easiest to understand.

Mutual Info is the change in purity (that is, the entropy) between the parent node and the weighted average of the purities of the child nodes. The weighted average is based on the number of records at each child node.

Normalized Mutual Info (the default) is the Mutual Info divided by the log (base 2) of the number of child nodes.

Gain Ratio is the Mutual Info divided by the entropy of the split while ignoring the label values.

Normalized Mutual Info and Gain Ratio give preference to attributes with few values.

The options offered for Regressors determines how columns are selected in the regression tree.

Variance determines how columns are selected in the regression tree. Variance selects the column producing the split that minimizes the within-node variance. Choosing Variance will produce a regression tree with mean predictors at the leaves.

Absolute Deviation selects the column producing the split that minimizes the within-node absolute deviation. Choosing Absolute Deviation will produce a regression tree with median predictors at the leaves.

Normalized Variance (the default) is variance divided by the log (base 2) of the number of child nodes.

Normalized Absolute Deviation is absolute deviation divided by the log (base 2) of the number of child nodes.

The Statistics Visualizer, accessible from the Viz Tools tab in the Data Destinations pane of the Tool Manager, shows a window with a series of small panels, one for each column listed in the Current Columns pane of Tool Manager. Only a restricted number of column panels can be shown at a time, so use the side scroll bars, or stretch the Statistics Visualizer window horizontally or vertically to see more column panels.

With the Statistics Visualizer you can see certain statistics, based on the number of records in the dataset given to Tool Manager. The format of the column panel varies according to the column type, and the number of distinct values that exist for that column. Columns are generally divided into two types: numeric and discrete, shown as box plots and histograms, respectively.

How to Read Statistics Visualizer

The box plots are drawn from numeric columns, each numeric column is made up of either integer, float, double or date values. Each box plot panel shows statistics about data from a single column, including the minimum, maximum, mean, median, and two quartiles (25th and 75th percentiles) of these numeric values. These values are shown as lines across a vertical bar in graduated shades of green, and the standard deviation of the population is shown as a +/- value. The quartiles are shown whenever there are fewer than 50,000 distinct values, (see Figure 1-31). If there are more than 50,000 distinct values in the column, the statistics are shown as a gray vertical bar.

The mean is the number found by adding the data in a column, then dividing by the number of records. The median is the middle number when numbers in a given column are arranged in order of size. The standard deviation is a measure of the dispersion of the data in a column.

Figure 1-31. Numeric Column Displayed by Statistics Visualizer

Histograms are drawn for discrete (or nominal) columns, each of which has non-numeric (string, bin, or enum) values (see Figure 1-32). The discrete column panel shows up to 100 distinct values, as well as a histogram of the number of instances of this distinct value. The default ordering of the discrete rows is by decreasing count, but you can use the View pulldown menu to select an alternative sorting. If there are 100 or fewer distinct categories, then the column panel also contains the count of distinct values.

Histograms are used whenever discrete values are to be shown, for example, yes/no values, or state names. Each box, whether a box plot or a histogram, shows the number of records in the data set, as “total vals”, and the number of distinct records represented in this particular box, as “distinct vals.”

Figure 1-32. Discrete Column Displayed by Statistics Visualizer

If you started the Statistics Visualizer from the icon, the main window allows only the File and Help pulldown menus to be used. For the main window to show all menus and controls, open a .statviz file. Use File > Open () to see a list of configuration files.

MineSet allows you to apply a sequence of operations to transform a data table. This sequence of transformations is recorded, and a specific transformation can be identified using the two Table History buttons at the bottom of the Data Transformation panel of the Tool Manager. Using these buttons you can see this sequence of steps, and go back if you made a mistake. When you click the left arrow button, the columns window shows the table as it appeared at an earlier step. Clicking the right arrow button returns the table to its current state.

Figure 1-33. Table History Buttons “Current view is” Field

Prev and Next Buttons

As you go back and forth using the Table History buttons to view earlier transformations, the Prev and Next fields (under the arrow buttons) help you keep track of where you are in the history of the table. For any table you view, the Prev: field tells you what the previous transformation was, and the Next: field tells you the next transformation.

Edit Prev. Op. Button

The Edit Prev. Op. button allows you to edit the operation shown in the Prev. field. (This button is not active when Current view is: 1 of some number, because that is the original table, with no previous changes.) When you click the Edit Prev. Op. button, the dialog box for the previous operation comes up, and you can make changes to that transformation. For example, if the previous transformation was binning columns, when you click Edit Prev. Op., the Bin Columns dialog box appears.

By changing a previous transformation, you could affect transformations you set up subsequent to the current one. For example, if you delete a column that you used in a subsequent binning operation, that binning operation becomes invalid. The Edit History button can help you avoid such problems.

Delete Ops. to End Button (Windows Only)

Whenever you back up in the history, Tool Manager's Data Destination pane (on the right side of the main window) becomes disabled (greyed out) because you are no longer at the end of the history. For the Data Destination pane to become enabled again, you must either go forward to the end of the history, or click the Delete Ops. to End button to delete all of the operations that you applied after the current view.

History of Operations Tab

When you click the History of Operations Tab (View History button on IRIX), the panels showing the current columns and data destination are replaced by a panel showing you the complete history of the Data Transformation table (Figure 1-34). Each version of the table appears as a box containing a list of the columns, linked by a box (indicating the operation performed on the table) to the next version of it.

Figure 1-34. View History Dialog Box (Windows)

The icons on the toolbar of the Windows version allow you to examine and perform various operations to the dataset with the Tool Manager. Clicking on an operation in the left pane, then clicking the Single Transformation and Data Destination View tab (View Single Ops/Dest button on IRIX) it presents the Tool Manager at the selected stage of operations (see Figure 1-34 and Figure 1-35). Clicking the History of Operations View tab (View History button on IRIX) returns the previous view.

Figure 1-35. View History Dialog Box (IRIX)

As with Edit Prev. Op, changing one operation usually affects (sometimes invalidates) subsequent operations in the history. You can select a specific operation to edit, add, or view. The View History dialog warns you when changes affect the history, shows you the new history.

The row of buttons beneath the diagram window in the IRIX version of the View History panel allows you to change the size and orientation of the diagram.

Tool Manager is the graphical user interface for specifying the configuration file, data file and tools to be used. General operation of the Tool Manager is discussed in this section. Refer to the MineSet Enterprise Edition User's Guide for Windows for directions on how to use it.

Tool Manager runs on your MineSet client. The process typically follows this path:

The Tool Manager Preferences dialog box allows you to set the following options:

A training set is a table containing attributes, one of which is designated as the class label. The label is the attribute for which you are trying to generate a predictive model.

The goal in this example is to predict the type of an iris flower (iris-setosa, iris-versicolor, or iris-virginica) given as descriptive attributes its sepal length, sepal width, petal length and petal width. Figure 1-36 shows several records from a sample training set.

Figure 1-36. Sample Records From a Training Set

Once a model is built, it can predict the label value for new records. These new records must be in a table that has all the attributes used by the model with the same name and type as they were in the training set. The table need not contain the label attribute. If it exists, it is ignored during prediction.

The Tree Visualizer is a graphical interface that displays data as a three-dimensional “landscape.” It presents your data as hierarchical blocks ( nodes) and bars with disks through which you can dynamically navigate, viewing part, or all, of the dataset. Using the Tree Visualizer is detailed in the MineSet Enterprise Edition User's Guide for Windows .

The Tree Visualizer displays quantitative and relational characteristics of your data by showing them as hierarchically connected nodes. Each node contains bars whose height, color and disk correspond to aggregations of data values. The edges (displayed as lines) connecting nodes show the relationship of one set of data to its subsets.

Values in subgroups can be summed and displayed automatically in the next higher level. The base under the bars can provide information about the aggregate value of all the bars. Bars representing negative values are shown below the top of the base. You can see negative value bars more clearly by disabling the base height (see “Tree Visualizer Display Menu ”, or the MineSet Enterprise Edition Interface Guide,“Creating Data and Configuration Files for the Tree Visualizer”).

Tree Visualizer Options

Clicking the Tool Options button causes a new dialog box to be displayed (Figure 1-37 and Figure 1-38). This lets you change some of the Tree Visualizer options from their default values.

Figure 1-37. Tree Visualizer's Configuration Options Dialog Box (Windows)

In the Windows version, the top of the dialog box shows three tabs: Bars, Node Bases and Disks. Each of these tabs reveals a series of options that allow you to configure the details of your visualization. To understand about choosing colors, refer to “Color Selection”. To specify the Heights portion of each dialog box, refer to “ Normalize Heights” next.

Figure 1-38. Tree Visualizer's Configuration Options Dialog Box (IRIX)

In the IRIX version, the top of the dialog box has three columns: Bars, Node Bases, and Disks. All the options are accessible from this one dialog box.

Normalize Heights

This option lets you normalize heights across each level of the hierarchy (or across all levels) of bars, node bases, and disks. Normalizing the heights determines the maximum value of the height variable; it normalizes all values relative to that height. Thus, if the maximum value is 30.0, and the maximum bar height was set to 1.0 (in arbitrary units), a value of 15.0 would be mapped to a value of 0.5.

Normalizing across each level independently normalizes each level of the hierarchy. This option is most useful if data has been summed up the hierarchy, and prevents the top level of the hierarchy from dwarfing items at the lowest level. Normalizing across all levels normalizes everything together, regardless of the level in the hierarchy. If neither box is checked for bars, no normalization takes place.

Node Bases are normalized independently of Bars. If no boxes are checked, the same normalization method used for bars is used for node bases, although the values are normalized independently.

If disks are present and normalize with bars is checked, the disks are normalized in conjunction with the bars: a disk and a bar representing the same value have the same height. If one of the other normalize boxes is checked in the Disks column, disks are normalized independently of the bars: the highest disk and the tallest bar have the same height, regardless of the actual values represented by them.

Max/Scale Heights

This option lets you specify the height of the tallest bars and node bases. The default is 1.0 (in arbitrary units). If after looking at the view, you see that the heights are too low or too high, use this field to adjust them. For example, entering 2 in the field causes all bars to be doubled in height; entering .5 makes all bars half as big.

If normalization was specified, this value represents the height of the tallest bar or base. If normalization was not specified, all values are scaled by this amount. The latter can be useful when comparing views of two different datasets.

Filter out % shortest

This option lets you filter out nodes containing only short bars. First, the tallest bar in the scene is calculated (if heights are normalized by level, then the tallest bar in each level). Then only those nodes that contain at least one bar that is the appropriate percentage of the tallest bar are shown. For example, if you enter 5% in this field, then only those nodes containing at least one bar that is at least 5% of the height of the tallest bar are shown. (Also shown are ancestors of such bars). This option is intended as a coarse way to filter out small, uninteresting nodes. It is not intended as an exact mechanism of identifying specific nodes of a certain value. Use of this option can accelerate the rendering of slow, complex scenes, or reduce clutter resulting from many bars near zero height.

Although small nodes are filtered out, they are nonetheless counted in any cumulation up the hierarchy.

Height Aggregation

By default, the height of the bars of the parent node is the sum of the height of all the bars of the children; however, these heights can be average, max, min, count, or any of the values that appear. This aggregation can be used for the values of the bar heights, base heights, and disk heights.

Colors

This set of options lets you

• specify the list of colors to use

• specify the kind of mapping

• map colors to bars, node bases, and disks

To use these Colors options, you must have mapped a column to the Color - Bar, Color - Disk, or Color - Base requirements of the Data Destination panel. See “Color Selection” for a more detailed explanation of how to choose and change colors.

Color list to use lets you specify the color list using the + button next to the color list label. This brings up a color editor that lets you specify a color to be added to the list.

Kind of mapping lets you specify whether the color change that is shown in the graphic display is Continuous or Discrete. If you choose Continuous, the color values (of the bars, node bases, or disks) shift gradually between the colors entered in the Color list to use field as a function of the values that are mapped to those colors in the Color mapping field. If you choose Discrete, the colors change only at the specified boundaries.

Color mapping lets you specify values to which the colors are mapped.

Color Aggregation

By default, the values of the colors of the bars of the parent node are the sum of the values of all the bars of the children; however, these colors can be average, max, min, or any of the values that appear. This aggregation can be used for the values of the bar colors, base node colors, and disk colors.

Color by Key

This option lets you automatically color the bars by their key value. This option is ignored if another coloring was specified. If you specify no color list, or specify insufficient colors, additional colors are chosen at random. If extra colors are specified, they are ignored.

Make Fixed

By default, this option places all bars across one row. This option allows changing the number of rows or columns. If neither rows nor columns are selected, or the number is set to 0, then neither rows nor columns are fixed, and the closest approximation to a square is displayed.

Message

This option lets you type in any message you want. The message statement specifies the message displayed when the pointer is moved over an object or when an object is selected. By default, the same message is used for the base as for the bars. If no message is specified, a default message containing the names and values of all the columns is used.

The format of the message must match the type of data being used:

• Strings must use %s.

• Ints must use integer formats (like %d).

• Floats and doubles must used floating-point formats (like %f).

For a detailed description of the message field, see “Message Statements” in the chapter “Creating Data and Configuration Files for the Tree Visualizer” in the MineSet Enterprise Edition Interface Guide .

Execute and Base Execute

These options let you type in a command that is executed when double-clicking on a bar or base. If only the Execute field is filled in, it applies to both bars and bases. If both are filled in, Execute applies to bars, and Base Execute applies to bases. The format is similar to the message statement. If no execute statement appears, double-clicking has no effect.

For a detailed description of the Execute field, see “Execute Statement” in the chapter “Creating Data and Configuration Files for the Tree Visualizer” in the MineSet Enterprise Edition Interface Guide .

Sky Color

You can specify either one or two colors. If only one color is specified, the sky is solid. If two colors are specified, the sky is shaded between the colors. When specifying two colors, the first color is for the top of the sky, the second for the bottom.

Ground Color

You can specify either one or two colors. If only one color is specified, the ground is solid. If two colors are specified, the ground is shaded between the colors. For the ground, the first color is for the far horizon, the second is for the near ground.

Base Label Color

You can specify the color of the labels on the front of the bases.

Bar Label Color

You can specify the color of the labels on the front of the bars.

Line Color

You can specify the color of the lines connecting the bases.

Sort Order

If you select the Sort by Key check box, the nodes in the display are in sorted order. The menu next to the check box lets you specify whether to sort in ascending or descending order.

Resetting the Tool Options

If, after you have made changes to the Tool Options dialog box, you want to reset the values of all options to their default values, click the Reset Options button.

Saving the New Tool Options

Once you have finished making changes to the Tool Options dialog box, click OK to return to the Tool Manager's main screen.

You can access all of the Tree Visualizer's functions using the five pulldown menus labeled File, Show, Display, Go, and Help. The File menus is the same for most MineSet tools, see “File Menu”.

View Menu

The View menu (Show menu on IRIX systems) contains four options: Overview, Search Panel, Filter Panel and Marks Panel. Each of these options brings up another dialog box for interacting with the data.

The Search Panel

Select Search in the View (or Show) menu to bring up a dialog box that lets you specify criteria to search for objects (Figure 1-39 and Figure 1-40).

Figure 1-39. Tree Visualizer's Search Dialog Box (Windows)

You can specify the parts of the hierarchy to be searched. By default, the whole hierarchy is searched. To limit the levels searched, select a relational operator (such as <=) from the option menu that lets you specify the operand for the level. Then use the Level slider to select the level to be searched. Level 0 is the root of the hierarchy, level 1 is the level below that, and so forth. To search the root and the two levels below that, for example, choose <= 2.

You can also choose whether to search the bars or the bases.

The Hierarchy field lets you specify nodes to search. Below the Hierarchy field are fields that let you specify search criteria for individual columns (defined in the Current Columns: window of the Tool Manager's Table Processing pane.

To specify whether a search is case-sensitive, click the Ignore Case In Searches check box, in the Search panel. For example, if this toggle is on (a check mark appears on that button), the string “hello” is the same as “HellO.”

The check box labeled Treat Nulls as Zeros defaults to off, in which case, comparisons involving nulls cannot return TRUE in a search. If it is on, nulls are treated as equal to zero.

When searching through bars, the default is that all bars are searched. To search only a specific list of bars, you must select them. The Set All button turns on all bars; this is useful if most of the bars are to be searched, and only a few are to be turned off. The Clear button turns off all bars. If no bar is selected, the bar list is ignored, and all bars are searched.

Figure 1-40. Tree Visualizer's Search Dialog Box (IRIX)

To search for numeric values, enter the value, and select a relational operation (=, !=, >, <, >=, <=). To search for alphanumeric values, enter the string for which you want to search. You can use any of three types of string comparisons:

• “Contains” indicates that it contains the appropriate string. For example, California contains the strings Cal and forn.

• “Equals” requires the strings to match exactly.

• “Matches” allows wildcards:

  • An asterisk (*) represents any number of characters.

  • A question mark (?) represents one character.

  • Square braces ([ ]) enclose a list of characters to match.

  For example, California matches Cal*, Cal?fornia, and Cal[a-z]fornia.

In some cases (usually associated with binning in the Tool Manager), an option menu of values appears, instead of a text field. To ignore that variable, select Ignored in the Option menu. You can use relational operators (such as >=) with these options. This means that the specified value as well as subsequent ones are selected.

In addition to numeric and string comparison operations, you can specify Is Null, which is true if the value is null.

To the right of each search field is an additional option menu that lets you specify “And” or “Or” options. For example, you could specify “sales > 20 And < 40.” You can have any number of And or Or clauses for a given column, but cannot mix And and Or in a single column.

If different levels of the hierarchy are keyed by different types of data (for example, the top level is selected by strings, while the second level is selected by integers), then the “Hierarchy” search field is treated as a string and provides string operations, not number operations.

If the Ignore Case In Searches check box is checked, the comparisons of all string searches are case-insensitive.

The buttons across the bottom of the Search panel include some or all or the following:

• Search causes the search to be started. This button is automatically activated if the Enter key is pressed and the panel is active.

• Clear turns off all search spotlights and erases the values from the search fields.

• Next selects and zooms to the next matched object, in left-to-right order. After the last matched object is selected, clicking Next returns the view to the Home position. Next is valid only after a search that has found matches.

• Previous selects and zooms in the opposite order from that of the Next button.

• Select causes all objects that matched the search criteria to be selected. The Selection menu can then interact with these objects.

• Close closes the search window and turns off the search spotlights. If the Search panel is reopened, it is in the same state as it was before the last Close; clicking Search again repeats the last search.

  Figure 1-41. Sample Results of a Search in the Tree Visualizer

  
  
  

Once the search is complete, yellow spotlights highlight objects matching the search criteria (see Figure 1-41). To display information about an object under a yellow spotlight, move the pointer over that spotlight; the information appears in the upper left corner, under the label Pointer is over:. To select and zoom to an object under a yellow spotlight, left-click the spotlight; if you press the Control key while clicking, zooming does not occur.

Filter Panel

The Filter panel filters out selected information, thus fine-tuning the displayed hierarchy. You can use the Filter panel to emphasize specific information, or to shrink the amount of data for better performance. The Filter panel is similar for most MineSet tools, see “Filter Panel”.

The fields in the Tree Visualizer Filter dialog box follow the same conventions as that used in the Search dialog box. If the Ignore Case In Filters check box is checked, the comparisons of all string filters are case-insensitive.

If a node does not meet the filter criteria, has no bars that meet the criteria, and has no children that meet the criteria, the node is not shown. There can be, however, cases in which a specific object meets the filter criteria, but its ancestors up the tree do not. Also, other bars in the same node might not meet the criteria. Since position is important in interpreting context, it might not be good to eliminate those bars. Consequently, you are given an option of selecting one of three radio buttons that control how these objects should be drawn: Solid, Outline, and Hidden. Note, however, that if objects are drawn in a less solid form due to the Display Zeros or Display Null menu, they are displayed appropriately. For example, if Nulls are to be hidden, they are always hidden, regardless of the filter criteria.

The exception to this is when filtering to specific bars. In such a case, the other bars are eliminated and don't take up space, regardless of the radio button settings.

The Height Filter slider lets you filter out those nodes containing only short bars. The size of a value is shown as a percentage of the maximum height. First, the tallest bar in the scene is calculated (if heights are normalized by level, then the tallest bar in each level). Then only those nodes that contain at least one bar that is the appropriate percentage of the tallest bar are shown.

For example, if you enter 5% in this field, then only those nodes containing at least one bar that is at least 5% of the height of the tallest bar are shown. (Also shown are ancestors of such bars). This option is intended as a coarse way to filter out small, uninteresting nodes. It is not intended as an exact mechanism of identifying specific nodes of a certain value; use the search panel for that purpose. Use of this option can accelerate the rendering of slow, complex scenes, or reduce clutter resulting from many bars near zero height. You can also set this filtering option in the configuration file by using the Height Filter command.

Although small nodes are filtered out, they are nonetheless counted in any cumulation up the hierarchy.

The Depth slider, which is under the Height Filter slider, lets you display the hierarchy so that only a given number of levels are displayed at any given time. When you are at the top of the hierarchy, only the number of hierarchical levels specified by the slider is seen. The nodes in the rows are arranged to optimize their visibility. When navigating to nodes lower in the hierarchy, additional rows are made visible automatically. The nodes above them automatically adjust their locations to accommodate the newly added nodes; thus, some nodes might seem to move. Note that the overview shows all nodes in the hierarchy, not just the top nodes; thus, the layout of the overview might not match the layout of the main view. The X in the overview approximates the corresponding location in the main view; there is no exact mapping between the two layouts.

To start filtering, click the Filter button. If the Enter key is pressed while the panel is active, filtering automatically starts. To close the panel, click the Close button.