The association data converter requires:

The raw data file must be in the format shown in “About Transaction-Style Data”, in which there is one transaction ID and one item per row. Neither the transaction ID nor the item need be contiguous within the row of data, however. In addition, the records in the file must be of exactly the same length, and they must be grouped by transaction ID (though they need not be sorted).

The format file specifies the format of the raw data file to the association data converter. This format file must contain the following items in the order shown:

---

Note: Each column number is zero-based.

---

Most data files use only one field each for the item and the transaction identifier. For the example data listed above, assuming that each line is 80 characters wide (plus one for the end-of-line character), the format file would be:

S
80
1 5
0 5
1 74
6 74
0

This format file allows for a great deal of flexibility. For instance, there need be no separator between the transaction identifier and the item. The two may even be overlapping, as in:

bread  10012wheat  25
apple  10012fuji   25
banana 10012bunch  25

In this case, both the transaction ID and the item contain two fields of different lengths. If the total line width is 21 bytes, then the format file would be:

S
21     (21 bytes per line, not including end-of-line character)
2 7    (transaction ID is two fields, 7 bytes total)
7 5    (first transaction ID field is 5 bytes starting at column 7)
19 2   (second transaction ID field is 2 bytes starting at column 19)
2 13   (item is two fields, 13 bytes total)
0 6    (first item field is 6 bytes starting at column 0)
12 7   (second item field is 7 bytes starting at column 12)
0      (no description fields)

To find the association rules in your data, you must first convert the data into an intermediate binary format, which the Association Rules Generator uses to find the rules. This conversion step uses the raw data file and format file, and produces two new files:

The assoccvt program converts the data. Its usage is:

assoccvt [-ifile raw] [-ofile binary] format names 

where raw is the name of the raw data file, binary is the name of the produced binary data file, format is the name of the format file, and names is the name of the output names file. If the -ifile parameter is omitted, standard input is used instead. Similarly, if the -ofile parameter is omitted, the standard output is used instead.

The following command illustrates the use of the association data converter on the example file in /usr/lib/MineSet/assoccvt/examples. The file sing.data is an example of data in transaction format and has some simple grocery store transactions. Each line has a transaction number and the name of an item bought in that transaction. The format of this file is described by sing.format.

assoccvt -ifile sing.data -ofile sing.bin sing.format sing.names

To test whether the files for data conversion are correctly installed, run the preceding command from the shell command line. Then, using the UNIX diff command, compare the files created to those with the same name in /usr/lib/MineSet/assoccvt/examples. Compare sing.bin with /usr/lib/MineSet/assoccvt/examples/sing.bin, and compare sing.names with /usr/lib/MineSet/assoccvt/examples/sing.names.

The Association Rules Generator program, assocgen, requires:

Rules are generated by invoking the assocgen command, along with one or more parameters. Options fall into one of the following categories:

The -ropts string separates the two sets of options. This string is required if any options from the second set are used.

An example rule generation command line might be:

assocgen -prev 20 -tran sing.bin -ropts -names sing.names \-rout sing.rules

See “Rule Generation Options” and “Rule Restriction Options” for explanations of the parameters.

---

Note: In the assocgen program, support is called prevalence, and confidence is called predictability. Therefore, the parameters for the support and confidence thresholds are -prev and -pred.

---

The data listed in Table A-3 is an example of market basket data. This data can be found in the file /usr/lib/MineSet/assoccvt/examples/sing.data.

You can generate rules from the data in Table A-3 by using first assoccvt (see “Association Rules Generator Command-Line Operation”), and then running the assocgen command:

assocgen -prev 20 -tran sing.bin -ropts -names sing.names -rout sing.rules

The rules file that is output has the following format:

1    1    30.0000 100.00 40.00 Butter Eggs

The fields in each line correspond to:

The expected confidence is the frequency of occurrence of the RHS items. The difference between expected confidence and observed confidence is a measure of the increase in predictive power due to the presence of the LHS. Expected confidence gives an indication of what the confidence would be if there were no relationship between the items.

For a further description of the relationships between support, confidence, expected confidence, and lift, see Chapter 2, “Using the Association Rules Tool.”

If the minimum support threshold is 20% (8 records out of 38 in the example below), and the default minimum confidence threshold is 50%, the assocgen program generates the set of rules shown in Table A-4.

The rules generated by assocgen can be loaded into MineSet as a flat file for further analysis and visualization. You can use the Scatter Visualizer, for example, to visualize the rules in the same way that the Tool Manager is used to visualize rules generated from tabular data.

MineSet 2.6
# Example schema for loading rules into MineSet

input {
        options backslash on;
        file “assoc.rules.data”;
        int `lhs size`;
        int `rhs size`;
        float support;
        float confidence;
        float `expected confidence`;
        string LHS;
        string RHS;
}