SPMFAn Open-Source Data Mining Library

Introduction

Algorithms

Download

Documentation

Datasets

FAQ

License

Contributors

Citations

Performance

Developers' guide

Forum

Mailing-list

Blog

 

-------------
We are hiring a postdoctoral researcher (details here) and a Ph.D. student.  Send your CV with cover letter to Prof. Fournier-Viger.
-------------

439094 visitors
since 2010



Documentation

This section provides examples of how to use SPMF to perform various data mining tasks.

If you have any question or if you want to report a bug, you can check the FAQ, post in the forum or contact me. You can also have a look at the various articles that I have referenced on the algorithms page of this website to learn more about each algorithm.

List of examples

Itemset Mining (Frequent Itemsets, Rare Itemsets, etc.)

High-Utility Pattern Mining

Association Rule Mining

Clustering

Sequential Pattern Mining

Sequential Rule Mining

Sequence Prediction (source code version only)

Periodic pattern mining

Text Mining

Time Series Mining

Classification

Tools

Example 1 : Mining Frequent Itemsets by Using the Apriori Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Apriori" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Apriori contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestApriori_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is Apriori?

Apriori is an algorithm for discovering frequent itemsets in transaction databases. It was proposed by Agrawal & Srikant (1993).

What is the input of the Apriori algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the Apriori algorithm?

Apriori is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if Apriori is run on the previous transaction database with a minsup of 40 % (2 transactions), Apriori produces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Input file format

The input file format for Apriori is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The Apriori algorithm is an important algorithm for historical reasons and also because it is a simple algorithm that is easy to learn. However, faster and more memory efficient algorithms have been proposed. If efficiency is required, it is recommended to use a more efficient algorithm like FPGrowth instead of Apriori. You can see a performance comparison of Apriori, FPGrowth, and other frequent itemset mining algorithms by clicking on the "performance" section of this website.

Implementation details

In SPMF, there is also an implementation of Apriori that uses a hash-tree as an internal structure to store candidates. This structure provide a more efficient way to count the support of itemsets. This version of Apriori is named "Apriori_with_hash_tree" in the GUI of SPMF and the command line. For the source code version, it can be run by executing the test file MainTestAprioriHT_saveToFile.java. This version of Apriori can be up to twice faster than the regular version in some cases but it uses more memory. This version of Apriori has two parameters: (1) minsup and (2) the number of child nodes that each node in the hash-tree should have. For the second parameter, we suggest to use the value 30.

Where can I get more information about the Apriori algorithm?

This is the technical report published in 1994 describing Apriori.

R. Agrawal and R. Srikant. Fast algorithms for mining association rules in large databases. Research Report RJ 9839, IBM Almaden Research Center, San Jose, California, June 1994.

You can also read chapter 6 of the book "introduction to data mining" which provide a nice and easy to understand introduction to Apriori.

Example 2 : Mining Frequent Itemsets by Using the AprioriTid Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Apriori_TID" algorithm , (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Apriori_TID contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestAprioriTID_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is AprioriTID?

AprioriTID is an algorithm for discovering frequent itemsets (groups of items appearing frequently) in a transaction database. It was proposed by Agrawal & Srikant (1993).

AprioriTID is a variation of the Apriori algorithm. It was proposed in the same article as Apriori as an alternative implementation of Apriori. It produces the same output as Apriori. But it uses a different mechanism for counting the support of itemsets.

What is the input of the AprioriTID algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the AprioriTID algorithm?

AprioriTID is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if AprioriTID is run on the previous transaction database with a minsup of 40 % (2 transactions), AprioriTID produces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Input file format

The input file format used by AprioriTID is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The Apriori and AprioriTID algorithms are important algorithms for historical reasons and also because they are simple algorithms that are easy to learn. However, faster and more memory efficient algorithms have been proposed. For efficiency, it is recommended to use more efficient algorithms like FPGrowth instead of AprioriTID or Apriori. You can see a performance comparison of Apriori, AprioriTID, FPGrowth, and other frequent itemset mining algorithms by clicking on the "performance" section of this website.

Implementation details

There are two versions of AprioriTID in SPMF. The first one is called AprioriTID and is the regular AprioriTID algorithm. The second one is called AprioriTID_Bitset and uses bitsets as internal structures instead of HashSet of Integers to represent sets of transactions IDs. The advantage of the bitset version is that using bitsets for representing sets of transactions IDs is more memory efficient and performing the intersection of two sets of transactions IDs is more efficient with bitsets (it is done by doing the logical AND operation).

Optional parameter(s)

This implementation allows to specify additional optional parameter(s) :

  • "show transaction ids?" (true/false) This parameter allows to specify that transaction ids of transactions containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #TID followed by a list of transaction ids (integers separated by space). For example, a line terminated by "#TID: 0 2" means that the pattern on this line appears in the first and the third transactions of the transaction database (transactions with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestAprioriTID_..._saveToFile .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run Apriori_TID contextPasquier99.txt output.txt 40% true
This command means to apply the algorithm on the file "contextPasquier99.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 40%, and that transaction ids should be output for each pattern found.

Where can I get more information about the AprioriTID algorithm?

This is the technical report published in 1994 describing Apriori and AprioriTID.

R. Agrawal and R. Srikant. Fast algorithms for mining association rules in large databases. Research Report RJ 9839, IBM Almaden Research Center, San Jose, California, June 1994.

You can also read chapter 6 of the book "introduction to data mining" which provide a nice and easy to understand introduction to Apriori.

Example 3 : Mining Frequent Itemsets by Using the FP-Growth Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FPGrowth_itemsets" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FPGrowth_itemsets contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFPGrowth_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is FPGrowth?

FPGrowth is an algorithm for discovering frequent itemsets in a transaction database. It was proposed by Han et al. (2000). FPGrowth is a very fast and memory efficient algorithm. It uses a special internal structure called an FP-Tree.

What is the input of the FPGrowth algorithm?

The input of FPGrowth is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the FPGrowth algorithm?

FPGrowth is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if FPGrowth is run on the previous transaction database with a minsup of 40 % (2 transactions), FPGrowth produces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Input file format

The input file format used by FPGrowth is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

There exists several algorithms for mining frequent itemsets. In SPMF, you can try for example Apriori, AprioriTID, Eclat, HMine, Relim and more. Among all these algorithms, FPGrowth is generally the fastest and most memory efficient algorithm. You can see a performance comparison by clicking on the "performance" section of this website.

Where can I get more information about the FPGrowth algorithm?

This is the journal article describing FPGrowth:

Jiawei Han, Jian Pei, Yiwen Yin, Runying Mao: Mining Frequent Patterns without Candidate Generation: A Frequent-Pattern Tree Approach. Data Min. Knowl. Discov. 8(1): 53-87 (2004)

You can also read chapter 6 of the book "introduction to data mining" which provide an easy to understand introduction to FPGrowth (but does not give all the details).

Example 4 : Mining Frequent Itemsets by Using the Relim Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Relim" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Relim contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestRelim.java" in the package ca.pfv.SPMF.tests.

What is Relim?

Relim is an algorithm for discovering frequent itemsets in a transaction database. Relim was proposed by Borgelt (2005). It is not a very efficient algorithm. It is included in SPMF for comparison purposes.

What is the input of the Relim algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the Relim algorithm?

Relim is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if Relim is run on the previous transaction database with a minsup of 40 % (2 transactions), Relim produces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Input file format

The input file format used by Relim is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

There exists several algorithms for mining frequent itemsets. Relim is not a very efficient algorithm. For efficiency, it is recommended to use FPGrowth for better performance. You can see a performance comparison by clicking on the "performance" section of this website.

Where can I get more information about the FPGrowth algorithm?

This is the conference article describing Relim:

Keeping Things Simple: Finding Frequent Item Sets by Recursive Elimination Christian Borgelt. Workshop Open Source Data Mining Software (OSDM'05, Chicago, IL), 66-70. ACM Press, New York, NY, USA 2005

Note that the author of Relim and collaborators have proposed extensions and additional optimizations of Relim that I have not implemented.

Example 5 : Mining Frequent Itemsets by Using the Eclat / dEclat Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Eclat" or "dEclat" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Eclat contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, to run respectively Eclat or dEclat, launch the file "MainTestEclat_saveToMemory.java" or "MainTestDEclat_bitset_saveToMemory.java"in the package ca.pfv.SPMF.tests.

What is Eclat ?

Eclat is an algorithm for discovering frequent itemsets in a transaction database. It was proposed by Zaki (2001). Contrarily to algorithms such as Apriori, Eclat uses a depth-first search for discovering frequent itemsets instead of a breath-first search.

dEclat is a variation of the Eclat algorithm that is implemented using a structure called "diffsets" rather than "tidsets".

What is the input of the Eclat algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the Eclat algorithm?

Eclat is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if Eclat is run on the previous transaction database with a minsup of 40 % (2 transactions), Eclat produces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

Each frequent itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Input file format

The input file format used by ECLAT is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

There exists several algorithms for mining frequent itemsets. Eclat is one of the best. But generally, FPGrowth is a better algorithm. You can see a performance comparison by clicking on the "performance" section of this website. Note that recently (SPMF v0.96e), the Eclat implementation was optimized and is sometimes faster than FPGrowth.

Nevertheless, the Eclat algorithm is interesting because it uses a depth-first search. For some extensions of the problem of itemset mining such as mining high utility itemsets (see the HUI-Miner algorithm), the search procedure of Eclat works very well.

Implementation details

In SPMF, there are four versions of ECLAT. The first one is named "Eclat" and uses HashSets of Integers for representing sets of transaction IDs (tidsets). The second version is named "Eclat_bitset" and uses bitsets for representing tidsets. Using bitsets has the advantage of generally being more memory efficient and can also make the algorithm faster depending on the dataset.

There is also two versions of dEclat, which utilizes a structure called diffsets instead of tidsets. The versions having diffsets implemented as HashSets of integers and the version having diffsets implemented as bitsets are respectively named "dEclat_bitset" and "dEclat"

Optional parameter(s)

This implementation allows to specify additional optional parameter(s) :

  • "show transaction ids?" (true/false) This parameter allows to specify that transaction ids of transactions containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #TID followed by a list of transaction ids (integers separated by space). For example, a line terminated by "#TID: 0 2" means that the pattern on this line appears in the first and the third transactions of the transaction database (transactions with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestEclat_..._saveToFile .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:

java -jar spmf.jar run Eclat contextPasquier99.txt output.txt 40% true
This command means to apply the algorithm on the file "contextPasquier99.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 40%, and that transaction ids should be output for each pattern found.

Where can I get more information about the Eclat algorithm?

Here is an article describing the Eclat algorithm:

Mohammed Javeed Zaki: Scalable Algorithms for Association Mining. IEEE Trans. Knowl. Data Eng. 12(3): 372-390 (2000)

Here is an article describing the dEclat variation:

Zaki, M.J., Gouda, K.: Fast vertical mining using diffsets. Technical Report 01-1, Computer Science Dept., Rensselaer Polytechnic Institute (March 2001) 10

Example 6 : Mining Frequent Itemsets by Using the HMine Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HMine" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 0.4 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HMine contextPasquier99.txt output.txt 0.4 in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHMine.java" in the package ca.pfv.SPMF.tests.

What is H-Mine ?

H-Mine is an algorithm for discovering frequent itemsets in transaction databases, proposed by Pei et al. (2001). Contrarily to previous algorithms such as Apriori, H-Mine uses a pattern-growth approach to discover frequent itemsets.

What is the input of the H-Mine algorithm?

The input of H-Mine is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the H-Mine algorithm?

H-Mine is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if H-Mine is run on the previous transaction database with a minsup of 40 % (2 transactions), H-Mine produces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Performance

There exists several algorithms for mining frequent itemsets. H-Mine is claimed to be one of the best by their author. The implementation offered in SPMF is well-optimized.

Input file format

The input file format used by H-Mine is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Where can I get more information about the H-Mine algorithm?

Here is an article describing the H-Mine algorithm:

J. Pei, J. Han, H. Lu, S. Nishio, S. Tang, and D. Yang. "H-Mine: Fast and space-preserving frequent pattern mining in large databases". IIE Transactions, Volume 39, Issue 6, pages 593-605, June 2007, Taylor & Francis.

Example 7 : Mining Frequent Itemsets by Using the FIN Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FIN" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FIN contextPasquier99.txt output.txt 0.4 in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFIN.java" in the package ca.pfv.SPMF.tests.

What is FIN?

FIN is a very recent algorithm (2014)for discovering frequent itemsets in transaction databases, proposed by Deng et al. (2014). It is very fast.

This implementation is very faithful to the original. It was converted from the original C++ source code provided by Deng et al, and only contains some minor modifications.

What is the input of the FIN algorithm?

The input of FIN is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the FIN algorithm?

FIN is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if FIN is run on the previous transaction database with a minsup of 40 % (2 transactions), FINproduces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Performance

There exists several algorithms for mining frequent itemsets. FIN is claimed to be one of the best, and is certainly one of the top algorithms available in SPMF. The implementation is well optimized and faithful to the original version (it was converted from C++ to Java with only minor modifications).

Input file format

The input file format used by FIN is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Where can I get more information about the FIN algorithm?

Here is an article describing the FIN algorithm:

Zhi-Hong Deng, Sheng-Long Lv: Fast mining frequent itemsets using Nodesets. Expert Syst. Appl. 41(10): 4505-4512 (2014)

Example 8 : Mining Frequent Itemsets by Using the PrePost / PrePost+ Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "PrePost" or "PrePost+" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run PrePost contextPasquier99.txt output.txt 0.4 in a folder containing spmf.jar and the example input file contextPasquier99.txt
    or
    java -jar spmf.jar run PrePost+ contextPasquier99.txt output.txt 0.4 in a folder containing spmf.jar and the example input file contextPasquier99.txt
  • If you are using the source code version of SPMF, launch the file "MainTestPrePost.java" or "MainTestPrePost+.java" in the package ca.pfv.SPMF.tests.

What is PrePost / PrePost+?

PrePost is a very recent algorithm (2012) for discovering frequent itemsets in transaction databases, proposed by Deng et al. (2012).

PrePost+ is a variation designed by Deng et al. (2015). It is reported to be faster than PrePost. Both implementations are offered in SPMF.

These implementations are faithful to the original. They were converted from the original C++ source code provided by Deng et al, and only contains some minor modifications.

What is the input of the PrePost / PrePost+ algorithms?

The input of PrePost and PrePost+ is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the PrePost / PrePost+ algorithms?

PrePost and PrePost+ are algorithms for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if PrePost or PrePost+ are run on the previous transaction database with a minsup of 40 % (2 transactions), they produce the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Performance

There exists several algorithms for mining frequent itemsets. PrePost is claimed to be one of the best by their author. The PrePost+ algorithm by the same authors is supposed to be faster though (also offered in SPMF).

Input file format

The input file format used by PrePost is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Where can I get more information about the PrePost / PrePost+ algorithm?

Here is an article describing the PrePost algorithm:

Zhihong Deng, Zhonghui Wang, Jia-Jian Jiang: A new algorithm for fast mining frequent itemsets using N-lists. SCIENCE CHINA Information Sciences 55(9): 2008-2030 (2012)

And another describing PrePost+:

Zhihong Deng, Sheng-Dong Lv: PrePost + : An efficient N-lists-based algorithm for mining frequent itemsets via Children–Parent Equivalence pruning. Expert Systems and Applications, 42: 5424- 5432 (2015)

 

Example 9 : Mining Frequent Itemsets by Using the LCMFreq Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "LCMFreq" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run LCMFreq contextPasquier99.txt output.txt 0.4 in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestLCMFreq_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is LCMFreq?

LCMFreq is an algorithm of the LCM familly of algorithms for mining frequent itemsets. LCM is the winner of the FIMI 2004 competition. It is supposed to be one of the fastest itemset mining algorithm.

In this implementations,we have attempted to replicate LCM v2 used in FIMI 2004. Most of the key features of LCM have been replicated in this implementation (anytime database reduction, occurrence delivery, etc.). However, a few optimizations have been left out for now (transaction merging, removing locally infrequent items). They may be added in a future version of SPMF.

What is the input of the LCMFreq algorithm?

The input of LCMFreq is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the LCMFreq algorithm?

LCMFreq is an algorithm for discovering itemsets (group of items) occurring frequently in a transaction database (frequent itemsets). A frequent itemset is an itemset appearing in at least minsup transactions from the transaction database, where minsup is a parameter given by the user.

For example, if LCMFreq is run on the previous transaction database with a minsup of 40 % (2 transactions), LCMFreq produces the following result:

itemsets support
{1} 3
{2} 4
{3} 4
{5} 4
{1, 2} 2
{1, 3} 3
{1, 5} 2
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 2, 5} 2
{1, 3, 5} 2
{2, 3, 5} 3
{1, 2, 3, 5} 2

How should I interpret the results?

In the results, each itemset is annotated with its support. The support of an itemset is how many times the itemset appears in the transaction database. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter.

Performance

There exists several algorithms for mining frequent itemsets. LCMFreq is the winner of the FIMI 2004 competition so it is probably one of the best. In this implementation, we have attempted to replicate v2 of the algorithm. But some optimizations have been left out (transaction merging and removing locally infrequent items). The algorithm seems to perform well on sparse datasets.

Implementation details

In the source code version of SPMF, there are two versions of LCMFreq. The version "MainTestLCMFreq.java" keeps the result into memory. The version named "MainTestLCMFreq_saveToFile.java" saves the result to a file. In the graphical user interface and command line interface only the second version is offered.

Input file format

The input file format is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions.

1 #SUP: 3
2 #SUP: 4
3 #SUP: 4
5 #SUP: 4
1 2 #SUP: 2
1 3 #SUP: 3
1 5 #SUP: 2
2 3 #SUP: 3
2 5 #SUP: 4
3 5 #SUP: 3
1 2 3 #SUP: 2
1 2 5 #SUP: 2
1 3 5 #SUP: 2
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Where can I get more information about the LCMFreq algorithm?

Here is an article describing the LCM v2 familly of algorithms:

Takeaki Uno, Masashi Kiyomi and Hiroki Arimura (2004). LCM ver. 2: Efficient Mining Algorithms for Frequent/Closed/Maximal Itemsets. Proc. IEEE ICDM Workshop on Frequent Itemset Mining Implementations Brighton, UK, November 1, 2004

Example 10 : Mining Frequent Closed Itemsets Using the AprioriClose Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "AprioriClose" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run AprioriClose contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestAprioriClose1.java" in the package ca.pfv.SPMF.tests.

What is AprioriClose?

AprioriClose (aka Close) is an algorithm for discovering frequent closed itemsets in a transaction database. It was proposed by Pasquier et al. (1999).

What is the input of the AprioriClose algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the AprioriClose algorithm?

AprioriClose outputs frequent closed itemsets. To explain what is a frequent closed itemset, it is necessary to review a few definitions.

An itemset is an unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, consider the itemset {1, 3}. It has a support of 3 because it appears in three transactions (t1, t3 and t5) from the transaction database .

A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database, where minsup is a threshold set by the user. A frequent closed itemset is a frequent itemset that is not included in a proper superset having exactly the same support. The set of frequent closed itemsets is thus a subset of the set of frequent itemsets. Why is it interesting to discover frequent closed itemset ? The reason is that the set of frequent closed itemsets is usually much smaller than the set of frequent itemsets and it can be shown that no information is lost (all the frequent itemsets can be regenerated from the set of frequent closed itemsets - see Pasquier(1999) for more details).

If we apply AprioriClose on the previous transaction database with a minsup of 40 % (2 transactions), we get the following five frequent closed itemsets:

frequent closed itemsets support
{3} 4
{1, 3} 3
{2, 5} 4
{2, 3, 5} 3
{1, 2, 3, 5} 2
If you would apply the regular Apriori algorithm instead of AprioriClose, you would get 15 itemsets instead of 5, which shows that the set of frequent closed itemset can be much smaller than the set of frequent itemsets.

How should I interpret the results?

In the results, each frequent closed itemset is annotated with its support. For example, the itemset {2, 3, 5} has a support of 3 because it appears in transactions t2, t3 and t5. The itemset {2, 3, 5} is a frequent itemset because its support is higher or equal to the minsup parameter. Furthermore, it is a closed itemsets because it has no proper superset having exactly the same support.

Input file format

The input file format used by AprioriClose is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent closed itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example. The second line indicates the frequent itemset consisting of the item 1 and 3, and it indicates that this itemset has a support of 4 transactions.

3 #SUP: 4
1 3 #SUP: 3
2 5 #SUP: 4
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The AprioriClose algorithm is important for historical reasons because it is the first algorithm for mining frequent closed itemsets. However, there exists several other algorithms for mining frequent closed itemsets. In SPMF, it is recommended to use DCI_Closed or Charm instead of AprioriClose, because they are more efficient.

Implementation details

In SPMF, there are two versions of AprioriClose. The first version is named "AprioriClose" and is based on the "Apriori" algorithm. The second version is named "Apriori_TIDClose" and is based on the AprioriTID algorithm instead of Apriori (it uses tidsets to calculate support to reduce the number of database scans). Both version are available in the graphical user interface of SPMF. In the source code, the files "MainTestAprioriClose1.java" and"MainTestAprioriTIDClose.java" respectively correspond to these two versions.

Where can I get more information about the AprioriClose algorithm?

The following article describes the AprioriClose algorithm:

Nicolas Pasquier, Yves Bastide, Rafik Taouil, Lotfi Lakhal: Discovering Frequent Closed Itemsets for Association Rules. ICDT 1999: 398-416

Example 11 : Mining Frequent Closed Itemsets Using the DCI_Closed Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "DCI_Closed" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 2 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run DCI_Closed contextPasquier99.txt output.txt 2 in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestDCI_Closed.java" in the package ca.pfv.SPMF.tests.

What is DCI_Closed?

DCI_Closed is an algorithm for discovering frequent closed itemsets in a transaction database. DCI_Closed was proposed by Lucchese et al. (2004).

What is the input of the DCI_Closed algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the DCI_Closed algorithm?

DCI_Closed outputs frequent closed itemsets. To explain what is a frequent closed itemset, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, consider the itemset {1, 3}. It has a support of 3 because it appears in three transactions (t1, t3, t5) from the transaction database.

A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A frequent closed itemset is a frequent itemset that is not included in a proper superset having exactly the same support. The set of frequent closed itemsets is thus a subset of the set of frequent itemsets. Why is it interesting to discover frequent closed itemsets ? The reason is that the set of frequent closed itemsets is usually much smaller than the set of frequent itemsets and it can be shown that no information is lost (all the frequent itemsets can be regenerated from the set of frequent closed itemsets - see Lucchese (2004) for more details).

If we apply DCI_Closed on the transaction database with a minsup of 2 transactions, we get the following result:

frequent closed itemsets support
{3} 4
{1, 3} 3
{2, 5} 4
{2, 3, 5} 3
{1, 2, 3, 5} 2
If you compare this result with the output of a frequent itemset mining algorithm like Apriori, you would notice that only 5 closed itemsets are found by DCI_Closed instead of about 15 itemsets by Apriori, which shows that the set of frequent closed itemset can be much smaller than the set of frequent itemsets.

How should I interpret the results?

In the results, each frequent closed itemset is annotated with its support. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter. It is a closed itemsets because it has no proper superset having exactly the same support.

Input file format

The input file format used by DCI_Closed is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent closed itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example. The second line indicates the frequent itemset consisting of the item 1 and 3, and it indicates that this itemset has a support of 4 transactions.

3 #SUP: 4
1 3 #SUP: 3
2 5 #SUP: 4
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The DCI_Closed algorithm is one of the fastest algorithms for frequent closed itemset mining. The version in SPMF is optimized and very efficient. SPMF also offers other algorithms for frequent closed itemset mining such as Charm and AprioriClose. DCI_Closed and Charm are more efficient than AprioriClose.

Implementation details

In the source code version of SPMF, there are two versions of DCI_Closed. The first one uses HashSet to store the transaction ids. The second one is an optimized version that uses a bit matrix to store transactions ids, and also includes additional optimizations. The first version can be tested by running MainTestDCI_Closed.java and the second version by running MainTestDCI_Closed_Optimized.java. In the release version of SPMF, only the optimized version of DCI_Closed is available in the graphical user interface and command line interface.

Optional parameter(s)

This implementation allows to specify additional optional parameter(s) :

  • "show transaction ids?" (true/false) This parameter allows to specify that transaction ids of transactions containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #TID followed by a list of transaction ids (integers separated by space). For example, a line terminated by "#TID: 0 2" means that the pattern on this line appears in the first and the third transactions of the transaction database (transactions with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestDCI_Closed_Optimized .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:

java -jar spmf.jar run DCI_Closed contextPasquier99.txt output.txt 2 true
This command means to apply the algorithm on the file "contextPasquier99.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 2 transactions, and that transaction ids should be output for each pattern found.

Where can I get more information about the DCI_Closed algorithm?

Here is an article describing the DCI_Closed algorithm:

Claudio Lucchese, Salvatore Orlando, Raffaele Perego: DCI Closed: A Fast and Memory Efficient Algorithm to Mine Frequent Closed Itemsets. FIMI 2004

Example 12 : Mining Frequent Closed Itemsets Using the Charm / dCharm Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Charm_bitset"or"dCharm_bitset" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Charm_bitset contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF and want to respectively launch Charm or dCharm, then launch the file "MainTestCharm_bitset_saveToMemory.java" or "MainTestDCharm_bitset_saveToMemory.java"in the package ca.pfv.SPMF.tests.

What is Charm?

Charm is an algorithm for discovering frequent closed itemsets in a transaction database. It was proposed by Zaki (2002).

dCharm is a variation of the Charm algorithm that is implemented with diffsets rather than tidsets. It has the same output and input as Charm.

What is the input of the Charm / dCharm algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the Charm / dCharm algorithm?

Charm outputs frequent closed itemsets. To explain what is a frequent closed itemset, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions (t1, t3, t5) from the previous transaction database.

A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A frequent closed itemset is a frequent itemset that is not included in a proper superset having exactly the same support. The set of frequent closed itemsets is thus a subset of the set of frequent itemsets. Why is it interesting to discover frequent closed itemsets ? The reason is that the set of frequent closed itemsets is usually much smaller than the set of frequent itemsets and it can be shown that no information is lost by discovering only frequent closed itemsets (because all the frequent itemsets can be regenerated from the set of frequent closed itemsets - see Zaki (2002) for more details).

If we apply Charm on the previous transaction database with a minsup of 40 % (2 transactions), we get the following result:

frequent closed itemsets support
{3} 4
{1, 3} 3
{2, 5} 4
{2, 3, 5} 3
{1, 2, 3, 5} 2
If you compare this result with the output from a frequent itemset mining algorithm like Apriori, you would notice that only 5 closed itemsets are found by Charm instead of about 15 itemsets by Apriori, which shows that the set of frequent closed itemset can be much smaller than the set of frequent itemsets.

How should I interpret the results?

In the results, each frequent closed itemset is annotated with its support. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter. It is a closed itemset because it has no proper superset having exactly the same support.

Input file format

The input file format used by CHARM is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent closed itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example. The second line indicates the frequent itemset consisting of the item 1 and 3, and it indicates that this itemset has a support of 4 transactions.

3 #SUP: 4
1 3 #SUP: 3
2 5 #SUP: 4
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The Charm algorithm is an important algorithm because it is one of the first depth-first algorithm for mining frequent closed itemsets. In SPMF, Charm and DCI_Closed are the two most efficient algorithms for frequent closed itemset mining.

Optional parameter(s)

This implementation of Charm allows to specify additional optional parameter(s) :

  • "show transaction ids?" (true/false) This parameter allows to specify that transaction ids of transactions containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #TID followed by a list of transaction ids (integers separated by space). For example, a line terminated by "#TID: 0 2" means that the pattern on this line appears in the first and the third transactions of the transaction database (transactions with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestCharm..._SaveToFile .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:

java -jar spmf.jar run Charm_bitset contextPasquier99.txt output.txt 40% true
This command means to apply the algorithm on the file "contextPasquier99.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 40% transactions, and that transaction ids should be output for each pattern found.

Where can I get more information about the Charm algorithm?

This article describes the Charm algorithm:

Mohammed Javeed Zaki, Ching-Jiu Hsiao: CHARM: An Efficient Algorithm for Closed Itemset Mining. SDM 2002.

Here is an article describing the dCharm variation:

Zaki, M.J., Gouda, K.: Fast vertical mining using diffsets. Technical Report 01-1, Computer Science Dept., Rensselaer Polytechnic Institute (March 2001) 10

Example 13 : Mining Frequent Closed Itemsets Using the LCM Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "LCM" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run LCM contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF and want to launch LCM on the file contextPasquier99.txt, then launch the file "MainTestLCM_saveToMemory.java"in the package ca.pfv.SPMF.tests.

What is LCM?

LCM is an algorithm of the LCM familly of algorithms for mining frequent closed itemsets. LCM is the winner of the FIMI 2004 competition. It is supposed to be one of the fastest closed itemset mining algorithm.

In this implementations,we have attempted to replicate LCM v2 used in FIMI 2004. Most of the key features of LCM have been replicated in this implementation (anytime database reduction, occurrence delivery, etc.). However, a few optimizations have been left out for now (transaction merging, removing locally infrequent items). They may be added in a future version of SPMF.

What is the input of the LCM algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the LCM algorithm?

LCM outputs frequent closed itemsets. To explain what is a frequent closed itemset, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions (t1, t3, t5) from the previous transaction database.

A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A frequent closed itemset is a frequent itemset that is not included in a proper superset having exactly the same support. The set of frequent closed itemsets is thus a subset of the set of frequent itemsets. Why is it interesting to discover frequent closed itemsets ? The reason is that the set of frequent closed itemsets is usually much smaller than the set of frequent itemsets and it can be shown that no information is lost by discovering only frequent closed itemsets (because all the frequent itemsets can be regenerated from the set of frequent closed itemsets - see Zaki (2002) for more details).

If we apply LCM on the previous transaction database with a minsup of 40 % (2 transactions), we get the following result:

frequent closed itemsets support
{3} 4
{1, 3} 3
{2, 5} 4
{2, 3, 5} 3
{1, 2, 3, 5} 2
If you compare this result with the output from a frequent itemset mining algorithm like Apriori, you would notice that only 5 closed itemsets are found by LCM instead of about 15 itemsets by Apriori, which shows that the set of frequent closed itemset can be much smaller than the set of frequent itemsets.

How should I interpret the results?

In the results, each frequent closed itemset is annotated with its support. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter. It is a closed itemset because it has no proper superset having exactly the same support.

Input file format

The input file format used by LCM is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent closed itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example. The second line indicates the frequent itemset consisting of the item 1 and 3, and it indicates that this itemset has a support of 4 transactions.

3 #SUP: 4
1 3 #SUP: 3
2 5 #SUP: 4
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

There exists several algorithms for mining closed itemsets. LCM is the winner of the FIMI 2004 competition so it is probably one of the best. In this implementation, we have attempted to replicate v2 of the algorithm. But some optimizations have been left out (transaction merging and removing locally infrequent items). The algorithm seems to perform very well on sparse datasets. According to some preliminary experiments, it can be faster than Charm, dCharm and DCI_closed on sparse datasets, but may perform less well on dense datasets.

Implementation details

In the source code version of SPMF, there are two versions of LCM. The version "MainTestLCM.java" keeps the result into memory. The version named "MainTestLCM_saveToFile.java" saves the result to a file. In the graphical user interface and command line interface only the second version is offered.

Where can I get more information about the LCM algorithm?

This article describes the LCM algorithm:

Here is an article describing the LCM v2 familly of algorithms:

Takeaki Uno, Masashi Kiyomi and Hiroki Arimura (2004). LCM ver. 2: Efficient Mining Algorithms for Frequent/Closed/Maximal Itemsets. Proc. IEEE ICDM Workshop on Frequent Itemset Mining Implementations Brighton, UK, November 1, 2004

Example 14 : Mining Frequent Closed Itemsets Using the FPClose Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FPClose"algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FPClose contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF and want to launch FPClose on the example input file contextPasquier99.txt, then launch the file "MainTestFPClose_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is FPClose?

FPClose is an algorithm of the FPGrowth familly of algorithms, designed for mining frequent closed itemsets. FPClose is supposed to be one of the fastest closed itemset mining algorithm.

In this implementations,we have attempted to implement most of the optimizations proposed in the FPClose paper, except that we did not implement the triangular matrix from FPGrowth* and the local CFI trees. These optimizations may be added in a future version of SPMF.

What is the input of the FPClose algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the FPClose algorithm?

FPClose outputs frequent closed itemsets. To explain what is a frequent closed itemset, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions (t1, t3, t5) from the previous transaction database.

A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A frequent closed itemset is a frequent itemset that is not included in a proper superset having exactly the same support. The set of frequent closed itemsets is thus a subset of the set of frequent itemsets. Why is it interesting to discover frequent closed itemsets ? The reason is that the set of frequent closed itemsets is usually much smaller than the set of frequent itemsets and it can be shown that no information is lost by discovering only frequent closed itemsets (because all the frequent itemsets can be regenerated from the set of frequent closed itemsets - see Zaki (2002) for more details).

If we apply FPClose on the previous transaction database with a minsup of 40 % (2 transactions), we get the following result:

frequent closed itemsets support
{3} 4
{1, 3} 3
{2, 5} 4
{2, 3, 5} 3
{1, 2, 3, 5} 2
If you compare this result with the output from a frequent itemset mining algorithm like Apriori, you would notice that only 5 closed itemsets are found by FPClose instead of about 15 itemsets by Apriori, which shows that the set of frequent closed itemset can be much smaller than the set of frequent itemsets.

How should I interpret the results?

In the results, each frequent closed itemset is annotated with its support. For example, the itemset {2, 3 5} has a support of 3 because it appears in transactions t2, t3 and t5. It is a frequent itemset because its support is higher or equal to the minsup parameter. It is a closed itemset because it has no proper superset having exactly the same support.

Input file format

The input file format used by FPClose is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent closed itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example. The second line indicates the frequent itemset consisting of the item 1 and 3, and it indicates that this itemset has a support of 4 transactions.

3 #SUP: 4
1 3 #SUP: 3
2 5 #SUP: 4
2 3 5 #SUP: 3
1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

There exists several algorithms for mining closed itemsets. FPClose is one of the fastest in the FIMI 2004 competition so it is probably one of the best. In this implementation, we have attempted most of the optimizations. But some optimizations have been left out (local CFI trees and the triangular matrix of FPGrowth*). The algorithm seems to perform very well.

Implementation details

In the source code version of SPMF, there are two versions of FPClose. The version "MainTestFPClose_saveToMemory.java" keeps the result into memory. The version named "MainTestFPClose_saveToFile.java" saves the result to a file. In the graphical user interface and command line interface only the second version is offered.

Where can I get more information about the FPClose algorithm?

This article describes the FPClose algorithm:

Grahne, G., & Zhu, J. (2005). Fast algorithms for frequent itemset mining using fp-trees. Knowledge and Data Engineering, IEEE Transactions on, 17(10), 1347-1362.

Example 15 : Mining Frequent Maximal Itemsets by Using the FPMax Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FPMax" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FPMax contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • To run this example with the source code version of SPMF, launch the file "MainTestFPMax.java" in the package ca.pfv.SPMF.tests.

What is FPMax?

FPMax is an algorithm for discovering frequent maximal itemsets in a transaction database.

FPMax is based on the famous FPGrowth algorithm and includes several strategies for mining maximal itemsets efficiently while pruning the search space.

What is the input of the FPMax algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the FPMax algorithm?

FPMax outputs frequent maximal itemsets. To explain what is a frequent maximal itemset, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions (t1,t3, t5) from the previous transaction database.

A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A frequent closed itemset is a frequent itemset that is not included in a proper superset having the same support. A frequent maximal itemset is a frequent itemset that is not included in a proper superset that is a frequent itemset. The set of frequent maximal itemsets is thus a subset of the set of frequent closed itemsets, which is a subset of frequent itemsets. Why it is interesting to discover frequent maximal itemsets ? The reason is that the set of frequent maximal itemsets is usually much smaller than the set of frequent itemsets and also smaller than the set of frequent closed itemsets. However, unlike frequent closed itemsets, frequent maximal itemsets are not a lossless representation of the set of frequent itemsets (it is possible to regenerate all frequent itemsets from the set of frequent maximal itemsets but it would not be possible to get their support without scanning the database).

If we apply FPMax on the previous transaction database with a minsup of 40 % (2 transactions), we get the following result:

frequent maximal itemsets support
{1, 2, 3, 5} 2
This itemset is the only maximal itemsets itemsets and it has a support of 2 because it appears in two transactions.

How should I interpret the results?

In the results, each frequent maximum itemset is annotated with its support. For example, the itemset {1, 2, 3 5} is a maximal itemset having a support of 2 because it appears in transactions t3 and t5. The itemset {2, 5} has a support of 4 and is not a maximal itemset because it is included in {2, 3, 5}, which is a frequent itemset.

Input file format

The input file format used by FPMax is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a maximal itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example consisting of a single line. The only line here indicates the maximal itemset consisting of the item 1, item 2, item 3 and item 5. This lines indicates that this itemset has a support of 2 transactions.

1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The FPMax algorithm is a very efficient algorithm for maximal itemset mining. I have tried to implement all the optimizations in the paper and optimize the implementation. However, it may be possible to still optimize it a little bit.

Where can I get more information about the FPMax algorithm?

The FPMax algorithm is described in this thesis (in French language only):

Grahne, G., & Zhu, J. (2003, May). High performance mining of maximal frequent itemsets. In 6th International Workshop on High Performance Data Mining.

Example 16 : Mining Frequent Maximal Itemsets Using the Charm-MFI Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Charm_MFI" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Charm_MFI contextPasquier99.txt output.txt 40% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • To run this example with the source code version of SPMF, launch the file "MainTestCharmMFI.java" in the package ca.pfv.SPMF.tests.

What is Charm-MFI?

Charm-MFI is an algorithm for discovering frequent maximal itemsets in a transaction database.

Charm-MFI is not an efficient algorithm because it discovers maximal itemsets by performing post-processing after discovering frequent closed itemsets with the Charm algorithm (hence the name: Charm-MFI). A more efficient algorithm for mining maximal itemsets named FPMax is provided in SPMF.

Moreover, note that the original Charm-MFI algorithm is not correct. In SPMF, it has been fixed so that it generates the correct result.

What is the input of the Charm-MFI algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the Charm-MFI algorithm?

Charm-MFI outputs frequent maximal itemsets. To explain what is a frequent maximal itemset, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions (t1,t3, t5) from the previous transaction database.

A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A frequent closed itemset is a frequent itemset that is not included in a proper superset having the same support. A frequent maximal itemset is a frequent itemset that is not included in a proper superset that is a frequent itemset. The set of frequent maximal itemsets is thus a subset of the set of frequent closed itemsets, which is a subset of frequent itemsets. Why it is interesting to discover frequent maximal itemsets ? The reason is that the set of frequent maximal itemsets is usually much smaller than the set of frequent itemsets and also smaller than the set of frequent closed itemsets. However, unlike frequent closed itemsets, frequent maximal itemsets are not a lossless representation of the set of frequent itemsets (it is possible to regenerate all frequent itemsets from the set of frequent maximal itemsets but it would not be possible to get their support without scanning the database).

If we apply Charm-MFI on the previous transaction database with a minsup of 40 % (2 transactions), we get the following result:

frequent maximal itemsets support
{1, 2, 3, 5} 2
This itemset is the only maximal itemsets itemsets and it has a support of 2 because it appears in two transactions.

How should I interpret the results?

In the results, each frequent maximum itemset is annotated with its support. For example, the itemset {1, 2, 3 5} is a maximal itemset having a support of 2 because it appears in transactions t3 and t5. The itemset {2, 5} has a support of 4 and is not a maximal itemset because it is included in {2, 3, 5}, which is a frequent itemset.

Input file format

The input file format used by CHARM-MFI is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a maximal itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example consisting of a single line. The only line here indicates the maximal itemset consisting of the item 1, item 2, item 3 and item 5. This lines indicates that this itemset has a support of 2 transactions.

1 2 3 5 #SUP: 2

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Optional parameter(s)

This implementation of Charm_MFI allows to specify additional optional parameter(s) :

  • "show transaction ids?" (true/false) This parameter allows to specify that transaction ids of transactions containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #TID followed by a list of transaction ids (integers separated by space). For example, a line terminated by "#TID: 0 2" means that the pattern on this line appears in the first and the third transactions of the transaction database (transactions with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestCharmMFI_SaveToFile .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:

java -jar spmf.jar run Charm_MFI contextPasquier99.txt output.txt 40% true
This command means to apply the algorithm on the file "contextPasquier99.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 40% transactions, and that transaction ids should be output for each pattern found.

Performance

The Charm-MFI algorithm is not a very efficient algorithm because it finds frequent maximal itemsets by post-processing instead of finding them directly.

A more efficient algorithm for mining maximal itemsets named FPMax is provided in SPMF.

Where can I get more information about the Charm-MFI algorithm?

The Charm-MFI algorithm is described in this thesis (in French language only):

L. Szathmary (2006). Symbolic Data Mining Methods with the Coron Platform.

Example 17 : Mining Frequent Generator Itemsets Using the DefMe Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "DefMe" algorithm, (2) select the input file "contextZart.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run DefMe contextZart.txt output.txt 40% in a folder containing spmf.jar and the example input file contextZart.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestDefMe_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is DefMe?

DefMe is an algorithm proposed at PAKDD 2014 for discovering minimal patterns in set systems. If it is applied to itemset mining, it will discover frequent itemset generator. In SPMF, we have implemented it for this purpose.

DefMe is the our knowledge the only real depth-first search algorithm for mining generator itemsets (it does not need to use a hash table or store candidates). It is interesting to have a depth-first search algorithm since depth-first search algorithm are generally faster than Apriori-based algorithms.

Another important point about DefMe is that unlike Pascal, DefMe only find frequent generator itemsets rather than generating all frequent itemsets and identifying which one are generators.

What is the input of the DefMe algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextZart.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3, 5}
t5 {1, 2, 3, 5}

What is the output of the DefMe algorithm?

The output of the DefMe algorithm for a transaction database and a minimum support threshold minsup is the set of all frequent itemsets and their support, and a flag indicating which itemsets is a generator.

To explain what is a frequent itemset and a generator, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions from the database (t2, t3 and t5). A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A generator is an itemset X such that there does not exist an itemset Y strictly included in X that has the same support.

By running DefMe with the previous transaction database and a minsup of 40% (2 transactions), we obtain the following result:

itemsets support
{} 5
{1} 4
{2} 4
{3} 4
{5} 4
{1, 2} 3
{1, 3} 3
{1, 5} 3
{2, 3} 3
{2, 5} 4
{3, 5} 3
{1, 2, 3} 2
{1, 3, 5} 2

How should I interpret the results?

In the results, for each generator itemset found, its support is indicated. For example, the itemset {1,2,3} has a support of 2 because it appears in 2 transactions (t3 and t5).

Input file format

The input file format used by DefMe is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For instance, the first line indicates that the empty set is a generator having a support of 5 transactions. The second line indicates that the itemset {1} has a support of 4 transactions.

#SUP: 5
1 #SUP: 4
1 2 #SUP: 3
1 2 3 #SUP: 2
1 3 #SUP: 3
1 3 5 #SUP: 2
1 5 #SUP: 3
2 #SUP: 4
2 3 #SUP: 3
3 #SUP: 4
3 5 #SUP: 3
5 #SUP: 4

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The DefMe algorithm should be more efficient than Apriori-based algorithm such as Zart or Pascal. However, no performance comparison has been done by the authors of DefMe.

Where can I get more information about the Pascal algorithm?

The DefMe algorithm is described in this paper:

Arnaud Soulet, François Rioult (2014). Efficiently Depth-First Minimal Pattern Mining. PAKDD (1) 2014: 28-39

Example 18 : Mining Frequent Closed Itemsets and Identify Generators Using the Pascal Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Pascal" algorithm, (2) select the input file "contextZart.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Pascal contextZart.txt output.txt 40% in a folder containing spmf.jar and the example input file contextZart.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestPascal.java" in the package ca.pfv.SPMF.tests.

What is Pascal?

Pascal is an algorithm for discovering frequent itemsets and at the same time identify which ones are generators in a transaction database.

Pascal is an Apriori-based algorithm. It uses a special pruning property that can avoid counting the support of some candidate itemsets. This property is based on the fact that if an itemset of size k is not a generator, then its support is the support of the minimum support of its subsets of size k-1.

What is the input of the Pascal algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextZart.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3, 5}
t5 {1, 2, 3, 5}

What is the output of the Pascal algorithm?

The output of the Pascal algorithm for a transaction database and a minimum support threshold minsup is the set of all frequent itemsets and their support, and a flag indicating which itemsets is a generator.

To explain what is a frequent itemset and a generator, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions from the database (t2, t3 and t5). A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A generator is an itemset X such that there does not exist an itemset Y strictly included in X that has the same support.

By running Pascal with the previous transaction database and a minsup of 40% (2 transactions), we obtain the following result:

itemsets is a generator? support
{} yes 5
{1} yes 4
{2} yes 4
{3} yes 4
{5} yes 4
{1, 2} yes 3
{1, 3} yes 3
{1, 5} yes 3
{2, 3} yes 3
{2, 5} yes 4
{3, 5} yes 3
{1, 2, 3} yes 2
{1, 2, 5} no 3
{1, 3, 5} yes 2
{2, 3, 5} no 3
{1, 2, 3, 5} no 2

How should I interpret the results?

In the results, all frequent itemsets are shown. Each frequent itemset that is a generator is marked as such ("yes"). For each itemset, its support is indicated. For example, the itemset {1,2,3,5} has a support of 2 because it appears in 2 transactions (t3 and t5) and it is not a generator because it has a subset {1,2,3} that has the same support.

Input file format

The input file format used by Pascal is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. After, all the items, the keyword "#IS_GENERATOR:" appears, which is followed by a boolean indicating if the itemset is a generator (true) or not (false). For example, here is the output file for this example. The first line indicates the frequent itemset consisting of the item 1 and it indicates that this itemset has a support of 3 transactions and is a generator.

1 #SUP: 0 #IS_GENERATOR true
2 #SUP: 0 #IS_GENERATOR true
3 #SUP: 0 #IS_GENERATOR true
5 #SUP: 0 #IS_GENERATOR true
1 2 #SUP: 2 #IS_GENERATOR true
1 3 #SUP: 3 #IS_GENERATOR true
1 5 #SUP: 2 #IS_GENERATOR true
2 3 #SUP: 3 #IS_GENERATOR true
2 5 #SUP: 4 #IS_GENERATOR true
3 5 #SUP: 3 #IS_GENERATOR true
1 2 3 #SUP: 2 #IS_GENERATOR false
1 2 5 #SUP: 2 #IS_GENERATOR false
1 3 5 #SUP: 2 #IS_GENERATOR false
2 3 5 #SUP: 3 #IS_GENERATOR false
1 2 3 5 #SUP: 2 #IS_GENERATOR false

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

The Pascal algorithm should be more or less as efficient as Apriori since it is an Apriori-based algorithm. Pascal utilizes a pruning strategies that is supposed to make it faster by avoiding counting the support of some candidates. But to see really which one is better, experiments would need to be done to compare it.

Where can I get more information about the Pascal algorithm?

The Pascal algorithm is described in this paper:

Bastide, Y., Taouil, R., Pasquier, N., Stumme, G., & Lakhal, L. (2000). Mining frequent patterns with counting inference. ACM SIGKDD Explorations Newsletter, 2(2), 66-75.

Example 19 : Mining Frequent Closed Itemsets and Minimal Generators Using the Zart Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Zart" algorithm, (2) select the input file "contextZart.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 40% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Zart contextZart.txt output.txt 40% in a folder containing spmf.jar and the example input file contextZart.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestZart.java" in the package ca.pfv.SPMF.tests.

What is Zart?

Zart is an algorithm for discovering frequent closed itemsets and their corresponding generators in a transaction database.

Zart is an Apriori-based algorithm. Why is it useful to discover closed itemsets and their generators at the same time? One reason is that this information is necessary to generate some special kind of association rules such as the IGB basis of association rules (see the example for IGB for more information about IGB association rules).

What is the input of the Zart algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextZart.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3, 5}
t5 {1, 2, 3, 5}

What is the output of the Zart algorithm?

The output of the Zart algorithm for a transaction database and a minimum support threshold minsup is the set of all frequent closed itemsets and their support, and the associated generator(s) for each closed frequent itemset.

To explain what is a frequent closed itemset and a generator, it is necessary to review a few definitions.

An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3} has a support of 3 because it appears in three transactions from the database (t2, t3 and t5). A frequent itemset is an itemset that appears in at least minsup transactions from the transaction database. A frequent closed itemset is a frequent itemset that is not included in a proper superset having the same support. A generator Y of a closed itemset X is an itemset such that (1) it has the same support as X and (2) it does not have any subset having the same support.

By running Zart with the previous transaction database and a minsup of 40% (2 transactions), we obtain the following result:

itemsets support is closed? minimal generators
{} 5 yes {}
{1} 4 yes {1}
{2} 4 no
{3} 4 yes {3}
{5} 4 no
{1, 2} 3 no
{1, 3} 3 yes {1,3}
{1, 5} 3 no
{2, 3} 3 no
{2, 5} 4 yes {2}, {5}
{3, 5} 3 no
{1, 2, 3} 2 no
{1, 2, 5} 3 yes {1, 2}, {1, 5}
{1, 3, 5} 2 no
{2, 3, 5} 3 yes {2, 3}, {3, 5}
{1, 2, 3, 5} 2 yes {1, 2, 3}, {1, 3, 5}

How should I interpret the results?

In the results, all frequent itemsets are shown. Each frequent itemset that is a closed itemset is marked as such ("yes"). For each closed itemset, its support is indicated and its list of generators. For example, the itemset {1,2,3,5} has a support of 2 because it appears in 2 transactions (t3 and t5). It is a closed itemset because it has no proper superset having the same support. Moreover is has two generators: {1, 2, 3} and {1, 3, 5}. By definition, these generators have the same support as {1, 2, 3, 5}.

Another example. The itemset {1, 3, 5} is not closed and it has a support of 2. It is not closed because it has a proper superset {1, 2, 3, 5} that has the same support.

Input file format

The input file format used by Zart is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, containing two sections.

The first section starts with "======= List of closed itemsets and their generators ============" on the first line of the file. Then, each closed itemset is indicated on a single line as follows. A closed itemset is represented by a line starting with "CLOSED :" followed by the itemset itself and then the support of the itemset. An itemset is represented by a list of integers, where each integer represents an item and where integers (items) are separated by single spaces. The support of a closed itemset is indicated by an integer immediately following the special keyword "#SUP:" on the same line. The support is expressed as a number of transactions. On the lines immediately following a closed itemset, the keyword "GENERATORS :" appears. Then, on the immediately following line, the generators of the itemsets are listed, one per line. A generator is represented by the keyword "=" followed by the itemset representing the generator. If a generator is the empty set, then it is represented by the keyword EMPTYSET.

The second sections starts with "======= List of frequent itemsets ============" on a single line. Then all frequent itemsets are listed on the following lines, one per line. On each line, the keyword "ITEMSET :" appears followed by the items of the itemset. Each item is represented by an integer and it is followed by a single space. After, all the items, the special keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions.

For example, we show below the output file for the previous example.

======= List of closed itemsets and their generators ============
CLOSED :
EMPTYSET #SUP: 5
GENERATOR(S) :
EMPTYSET
CLOSED :
1 #SUP: 4
GENERATOR(S) :
1
CLOSED :
3 #SUP: 4
GENERATOR(S) :
3
CLOSED :
1 3 #SUP: 3
GENERATOR(S) :
1 3
CLOSED :
2 5 #SUP: 4
GENERATOR(S) :
2
5
CLOSED :
1 2 5 #SUP: 3
GENERATOR(S) :
1 2
1 5
CLOSED :
2 3 5 #SUP: 3
GENERATOR(S) :
2 3
3 5
CLOSED :
1 2 3 5 #SUP: 2
GENERATOR(S) :
1 2 3
1 3 5
======= List of frequent itemsets ============
ITEMSET : EMPTYSET #SUP: 5
ITEMSET : 1 #SUP: 4
ITEMSET : 2 #SUP: 4
ITEMSET : 3 #SUP: 4
ITEMSET : 5 #SUP: 4
ITEMSET : 1 2 #SUP: 3
ITEMSET : 1 3 #SUP: 3
ITEMSET : 2 3 #SUP: 3
ITEMSET : 1 5 #SUP: 3
ITEMSET : 2 5 #SUP: 4
ITEMSET : 3 5 #SUP: 3
ITEMSET : 1 2 3 #SUP: 2
ITEMSET : 1 2 5 #SUP: 3
ITEMSET : 1 3 5 #SUP: 2
ITEMSET : 2 3 5 #SUP: 3
ITEMSET : 1 2 3 5 #SUP: 2

In this example, the first lines of the first section indicates that the empty set is a closed itemset with a support of 5 and that it is the generator of itself. The following lines indicates that the itemset {3} is closed, has a support of 4 and is the generator of itself. The following lines indicates that the itemset {1, 3} is closed, has a support of 3 and that the itemset {1} is the only generator for that itemset. The following lines of this section indicates in the same way the remaining closed itemsets and their associated generators.

In the same example, the first lines of the second sections indicates that the empty set is a frequent itemset with a support of 5 transactions, that the itemset 1 is frequent with a support of 3 transactions and that the itemset {2} is frequent with a support of 4 transactions. In the same way, the following lines indicates all the other frequent itemsets.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Implementation details

In the source code version of SPMF, there are two versions of Zart. The version "MainTestZart.java" keeps the result into memory. The version named "MainTestZart_saveToFile.java" saves the result to a file. In the graphical user interface and command line interface only the second version is offered.

Performance

The Zart algorithm is not a very efficient algorithm because it is based on Apriori. If someone only want to discover closed itemsets and do not need the information about generators, then he should instead use DCI_Closed or Charm, which are more efficient for closed itemset mining. However, in some cases it is desirable to discover closed itemset and their corresponding generators (for example to generate IGB association rules). For these cases, Zart is an appropriate algorithm.

Where can I get more information about the Zart algorithm?

The Zart algorithm is described in this paper:

L. Szathmary, A. Napoli, S. O. Kuznetsov. ZART: A Multifunctional Itemset Mining Algorithm. Laszlo Szathmary, Amedeo Napoli, Sergei O. Kuznetsov In: CLA, 2007.

Example 20 : Mining Minimal Rare Itemsets

How to run this example?

  • If you are using the graphical interface, (1) choose the "AprioriRare" algorithm, (2) select the input file "contextZart.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup to 60% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run AprioriRare contextZart.txt output.txt 60% in a folder containing spmf.jar and the example input file contextZart.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestAprioriRare.java" in the package ca.pfv.SPMF.tests.

What is AprioriRare?

AprioriRare is an algorithm for mining minimal rare itemsets from a transaction database. It is an Apriori-based algorithm. It was proposed by Szathmary et al. (2007).

What is the input ?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextZart.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3, 5}
t5 {1, 2, 3, 5}

What is the output?

The output of AprioriRare is the set of minimal rare itemsets. To explain what it a minimal rare itemset, it is necessary to review a few definitions. An itemset is a unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset divided by the total number of transactions. For example, the itemset {1, 2} has a support of 60% because it appears in 3 transactions out of 5 in the previous database (it appears in t1, t2 and t5). A frequent itemset is an itemset that has a support no less than the minsup parameter. A minimal rare itemset is an itemset that is not a frequent itemset and that all its subsets are frequent itemsets.

For example, if we run AprioriRare algorithm with minsup = 60 % and the previous transaction database, we obtain the following set of minimal rare itemsets:

Minimal Rare Itemsets Support
{4} 20 %
{1, 3, 5} 40 %
{1, 2, 3} 40 %

Input file format

The input file format of AprioriRARE is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format of AprioriRARE is defined as follows. It is a text file, where each line represents a maximal rare itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example.

4 #SUP: 1
1 2 #SUP: 2
1 5 #SUP: 2

The output file here consists of three lines which indicates that the itemsets {4}, {1, 2} {1, 5} are perfectly rare itemsets having respectively a support of 1 transaction, 2 transactions and 2 transactions.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

AprioriRare is the only algorithm for minimal rare itemset mining offered in SPMF. Since it is based on Apriori, it suffers from the same fundamental limitations (it may generate too much candidates and it may generate candidates that do not appear in the database).

Where can I get more information about this algorithm?

The AprioriRare algorithm is described in this paper:

Laszlo Szathmary, Amedeo Napoli, Petko Valtchev: Towards Rare Itemset Mining. ICTAI (1) 2007: 305-312

Example 21 : Mining Perfectly Rare Itemsets Using the AprioriInverse Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "AprioriInverse" algorithm, (2) select the input file "contextInverse.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 0.1 % and maxsup of 60 % and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run AprioriInverse contextZart.txt output.txt 10% 60% in a folder containing spmf.jar and the example input file contextInverse.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestAprioriInverse.java" in the package ca.pfv.SPMF.tests.

What is AprioriInverse?

AprioriInverse is an algorithm mining perfectly rare itemsets. Why mining perfectly rare itemsets? One reason is that it is useful for generating the set of sporadic association rules.

What is the input?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextInverse.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3}
t5 {1, 2, 4, 5}

What is the output?

The output of AprioriInverse is the set of all perfectly rare itemsets in the database such that their support is lower than maxsup and higher than minsup. To explain what it a perfectly rare itemset, it is necessary to review a few definitions. An itemset is an unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset divided by the total number of transactions. For example, the itemset {1, 2} has a support of 60% because it appears in 3 transactions out of 5 in the previous database (it appears in t1, t2 and t5). A frequent itemset is an itemset that has a support no less than the maxsup parameter. A perfectly rare itemset (aka sporadic itemset) is an itemset that is not a frequent itemset and that all its proper subsets are also not frequent itemsets. Moreover, it has to have a support higher or equal to the minsup threshold.

By running the AprioriInverse algorithm with minsup = 0.1 % and maxsup of 60 % and this transaction database, we obtain the following set of perfectly rare itemsets (see Koh & Roundtree 2005 for further details):

Perfectly Rare Itemsets Support
{3} 60 %
{4} 40 %
{5} 60 %
{4, 5} 40 %
{3, 5} 20 %

Input file format

The input file format of AprioriInverse is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format of AprioriInverse is defined as follows. It is a text file, where each line represents a perfectly rare itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, we show below the output file for this example.

3 #SUP: 3
4 #SUP: 2
5 #SUP: 3
3 5 #SUP: 1
4 5 #SUP: 2

The output file here consists of five lines which indicate that the itemsets {3}, {4}, {5}, {3, 5}, {4, 5} are perfectly rare itemsets having respectively a support of 3, 2, 3 1 and 2 transactions.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

AprioriInverse is the only algorithm for perfectly rare itemset mining offered in SPMF. Since it is based on Apriori, it suffers from the same fundamental limitations (it may generate too much candidates and may generate candidates that do not appear in the database).

Where can I get more information about this algorithm?

The AprioriInverse algorithm is described in this p aper:

Yun Sing Koh, Nathan Rountree: Finding Sporadic Rules Using Apriori-Inverse. PAKDD 2005: 97-106

Example 22 : Mining Rare Correlated Itemsets Using the CORI Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "CORI" algorithm, (2) select the input file "contextPasquier99.txt", (3) set the output file name (e.g. "output.txt") (4) set maxsup = 0.8 % and minbond = 20 % and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run CORI contextPasquier99.txt output.txt 80% 20% in a folder containing spmf.jar and the example input file contextPasquier99.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestCORI_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is CORI?

CORI is an algorithm for mining rare correlated itemsets.

It is an extension of the ECLAT algorithm. It uses two measures called the support and the bond to evaluate if an itemset is interesting and should be output.

What is the input of the CORI algorithm?

The input is a transaction database (aka binary context) and a threshold named minsup (a value between 0 and 100 %).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 3 and 4. This database is provided as the file contextPasquier99.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 3, 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {1, 2, 3, 5}

What is the output of the CORI algorithm?

CORI is an algorithm for discovering itemsets (group of items) that are rare and correlated in a transaction database (rare correlated itemsets). A rare itemset is an itemset such that its support is no less than a minsup threshold set by the user. The support of an itemset is the number of transactions containing the itemset.

A correlated itemset is an itemset such that its bond is no less than a minbond threshold set by the user. The bond of an itemsets is the number of transactions containing the itemset divided by the number of transactions containing any of its items. The bond is a value in the [0,1] interval. A high value means a highly correlated itemset. Note that single items have by default a bond of 1.

For example, if CORI is run on the previous transaction database with a minsup = 80% and minbond = 20%, CORI outputs the following rare correlated itemsets:

itemsets bond support
{1} 1 3
{4} 1 1
{1, 4} 0.33 1
{3, 4} 0.25 1
{1, 3, 4} 0.25 1
{1, 2} 0.4 2
{1, 2, 3} 0.4 2
{1, 2, 5} 0.4 2
{1, 2, 3, 5} 0.4 2
{1, 3} 0.75 3
{1, 3, 5} 0.4 2
{1, 5} 0.4 2
{2, 3} 0.6 3
{2, 3, 5} 0.6 3
{3, 5} 0.6 3

Input file format

The input file format used by CORI is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 3 4
2 3 5
1 2 3 5
2 5
1 2 3 5

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format of CORI is defined as follows. It is a text file, where each line represents a correlated rare itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. After, all the items, the keyword "#BOND:" appears, which is followed by a double value indicating the bond of the itemset. For example, we show below the output file for this example.

1 #SUP: 3 #BOND: 1.0
4 #SUP: 1 #BOND: 1.0
4 1 #SUP: 1 #BOND: 0.3333333333333333
4 3 #SUP: 1 #BOND: 0.25
4 1 3 #SUP: 1 #BOND: 0.25
1 2 #SUP: 2 #BOND: 0.4
1 2 3 #SUP: 2 #BOND: 0.4
1 2 5 #SUP: 2 #BOND: 0.4
1 2 3 5 #SUP: 2 #BOND: 0.4
1 3 #SUP: 3 #BOND: 0.75
1 3 5 #SUP: 2 #BOND: 0.4
1 5 #SUP: 2 #BOND: 0.4
2 3 #SUP: 3 #BOND: 0.6
2 3 5 #SUP: 3 #BOND: 0.6
3 5 #SUP: 3 #BOND: 0.6

The output file here consists of 15 lines. Consider the last line. It indicates that the itemset {3, 5} is a rare correlated itemset having a support and bond of respectively 3 and 0.6.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Optional parameter(s)

This implementation allows to specify additional optional parameter(s) :

  • "show transaction ids?" (true/false) This parameter allows to specify that transaction ids of transactions containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #TID followed by a list of transaction ids (integers separated by space). For example, a line terminated by "#TID: 0 2" means that the pattern on this line appears in the first and the third transactions of the transaction database (transactions with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestCORI_SaveToFile .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:

java -jar spmf.jar run CORI contextPasquier99.txt output.txt 80% 20% true
This command means to apply the algorithm on the file "contextPasquier99.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for maxsup = 80%, minbond = 20%, and that transaction ids should be output for each pattern found.

Performance

CORI is the only algorithm for mining correlated rare itemsets offered in SPMF. The implementation is well optimized. It is a quite simple extension of the ECLAT algorithm.

Where can I get more information about this algorithm?

The CORI algorithm is described in this paper:

Bouasker, S., Yahia, S. B. (2015). Key correlation mining by simultaneous monotone and anti-monotone constraints checking. Proc. of the 2015 ACM Symposium on Applied Computing (SAC 2015), pp. 851-856.

Example 23 : Mining Closed Itemsets from a Data Stream Using the CloStream Algorithm

How to run this example?

  • This example is not available in the release version of SPMF.
  • To run this example with the source code version of SPMF, launch the file "MainTestCloStream.java" in the package ca.pfv.SPMF.tests.

What is CloStream?

CloStream is an algorithm for incrementally mining closed itemsets from a data stream. It was proposed by Yen et al. (2009).

Why is it useful? Because most closed itemset mining algorithms such as Charm, DCI_Closed and AprioriClose are batch algorithms. This means that if the transaction database is updated, we need to run the algorithms again to update the set of closed itemsets. If there is constant insertion of new transactions and the results need to be updated often, it may become very costly to use these algorithms. A stream mining algorithm like CloStream is specially designed to handle this situation. It assumes that each transaction in a database can only be read once and that new transaction appears regularly. Every time that a new transaction appear, the result is updated by CloStream.

What is the input of CloStream?

The input of CloStream is a stream of transactions. Each transaction is a set of items (symbols). For example, consider the following five transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2 and 4. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction. CloStream is an algorithm for processing a stream. This means that CloStream is allowed to read each transaction only once because a stream is assumed to be potentially infinite and coming at high speed.

Transaction ID Items
t1 {1,2 4}
t2 {2, 3, 5}
t3 {1, 2, 3, 5}
t4 {2, 5}
t5 {2, 3, 5}

What is the output of CloStream?

CloStream produces as output the set of closed itemsets contained in the transactions that it has seen until now. An itemset is an unordered set of distinct items. The support of an itemset is the number of transactions that contains the itemset. For example, the itemset {1, 2, 4} has a support of 1 because it only appear in t1. A closed itemset is an itemset that is not included in another itemset having the same support. For example, if we apply CloStream to the five following transactions, the final result is:
closed itemsets support
{} 5
{3} 4
{1, 3} 3
{1, 3, 4} 1
{2, 5} 4
{2, 3, 5} 3
{1, 2, 3, 5} 2

For example, the itemset {2, 3, 5} has a support of 3 because it appears in transactions t2, t4 and t5. It is a closed itemset because it has no proper superset having the same support.

Input and output file format

This is not applicable for this algorithm since it is designed for a stream of data (see the source code example referenced above to understand how to use this algorithm).

Performance

CloStream is a reasonably efficient algorithm. A limitation of this algorithm is that it is not possible to set a minimum support threshold. Therefore, if the number of closed itemsets is large, this algorithm may use too much memory. However, CloStream has the advantage of being very simple an easy to implement.

Where can I get more information about this algorithm?

The CloStream algorithm is described in this paper:

Show-Jane Yen, Yue-Shi Lee, Cheng-Wei Wu, Chin-Lin Lin: An Efficient Algorithm for Maintaining Frequent Closed Itemsets over Data Stream. IEA/AIE 2009: 767-776.

Example 24 : Mining Recent Frequent Itemsets from a Data Stream Using the estDec Algorithm

How to run this example?

  • This example is not available in the release version of SPMF.
  • To run this example with the source code version of SPMF, launch the file "MainTestEstDec_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is estDec?

estDec is an algorithm for mining recent frequent itemsets from a data stream. It was proposed by Chang et al. (2003).

Why is it useful? Because most itemset mining algorithms such as Apriori, FPGrowth and Eclat are batch algorithms. This means that if the input transaction database is updated, those algorithms need to be run again from zero to update the result, which is inefficient. Stream mining algorithms such as estDec are designed for discovering patterns in a stream (a potentially infinite sequence of transactions) and for updating the results incrementally after each new transaction. Stream mining algorithms assume that each transaction in a database can only be read once. The estDec algorithm is also interesting because it mines recent frequent itemsets, which means that it put more weight on recent transactions than on older transactions when searching from frequent itemsets. This allows estDec to learn new trends and to forgot older trends.

What is the input of estDec?

The input of estDec is a stream of transactions and a support threshold minsup. Each transaction is a set of items (symbols). For example, consider the following six transactions (t1, t2, ..., t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction. estDec is an algorithm for processing a stream. This means that estDec is allowed to read each transaction only once because a stream is assumed to be potentially infinite and coming at high speed.

Transaction ID Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 5}

What is the output of estDec?

estDec produces as output the set of recent frequent itemsets contained in the transactions that estDec has seen until now. It is said that estDec mines recent frequent itemsets because estDec utilizes a decay function so that estDec puts more weight on recent transactions than on older ones (frequency of an itemset). This allows estDec to learn new trends and to forgot older trends.

The output is a set of recent frequent itemsets. The support count of an itemset is the number of transactions that contains the itemset. For example, the itemset {1, 2, 4} has a support count of 1 because it only appear in t1. The support of an itemset is the number of transaction were the itemset appears divided by the total number of transactions seen until now. A frequent itemset is an itemset that has a support higher or equal to minsup.

The estDec algorithm is an approximate algorithm. It approximate the support of itemsets and returns itemsets that have an estimated support higher than minsup.

For example, consider the example MainTestEstDec_saveToFile.java. This example consists of loading the transactions from a file named "contextIGB.txt" provided in the SPMF distribution. Then, this example show how to save the result to a file. Here is the output:

3 5 #SUP: 0.5000519860383547
2 #SUP: 0.8333622131312072
1 2 3 #SUP: 0.33335643690463074
3 #SUP: 0.5000519860383547
1 4 #SUP: 0.3333448844517001
3 4 #SUP: 0.19334881331065332
1 3 5 #SUP: 0.33335643690463074
1 2 5 #SUP: 0.5000173262771588
2 5 #SUP: 0.8333622131312072
1 #SUP: 0.5000173262771588
2 3 5 #SUP: 0.5000519860383547
1 5 #SUP: 0.5000173262771588
2 3 #SUP: 0.5000519860383547
4 #SUP: 0.3333448844517001
1 4 5 #SUP: 0.3333448844517001
2 4 5 #SUP: 0.3333448844517001
1 2 #SUP: 0.5000173262771588
5 #SUP: 0.8333622131312072
1 3 #SUP: 0.33335643690463074
2 4 #SUP: 0.3333448844517001
1 2 4 #SUP: 0.3333448844517001
4 5 #SUP: 0.3333448844517001

For example, consider line 1. It indicates that the pattern {1, 2, 5} is a recent frequent itemsets with an estimated support of 50%

Note that we also provide a second example named MainTestEstDec_saveToMemory.java. This example shows how to process a set of transactions from memory instead of from a file and to keep the result into memory instead of saving the result to a file. This is especially useful, if you wish to integrate estDec into another Java program. The example also shows how to set the decay rate.

Input file format

The estDec algorithm can either take as input a stream in memory or read transactions from a file. The input file format of estDec is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 5

Output file format

The output file format of estDec is defined as follows. It is a text file, where each line represents a recently frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here are few lines of the output file for this example.

3 5 #SUP: 0.5000519860383547
2 #SUP: 0.8333622131312072
1 2 3 #SUP: 0.33335643690463074

The output file here consists of the first line indicates that the itemset {1, 2, 3} has an estimated support of 50 %.

Performance

estDec is a reasonably efficient algorithm.

Where can I get more information about this algorithm?

The estDec algorithm is described in this paper:

Joong Hyuk Chang, Won Suk Lee: Finding recent frequent itemsets adaptively over online data streams. KDD 2003: 487-492

Example 25 : Mining Recent Frequent Itemsets from a Data Stream Using the estDec+ Algorithm

How to run this example?

  • This example is not available in the release version of SPMF.
  • To run this example with the source code version of SPMF, launch the file "MainTestEstDecPlus_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is estDec+?

estDec+ is an algorithm for mining recent frequent itemsets from a data stream. It is an extension of estDec proposed by Chang et al. in 2005. The main difference with estDec is to use a compressed tree to maintain information about recent frequent itemsets, which may be more memory efficient in some cases but may decrease accuracy. Note that the version of estDec+ implemented here is based on the 2014 paper by Chang et al.

Why is it useful? Because most itemset mining algorithms such as Apriori, FPGrowth and Eclat are batch algorithms. This means that if the input transaction database is updated, those algorithms need to be run again from zero to update the result, which is inefficient. Stream mining algorithms such as estDec+ are designed for discovering patterns in a stream (a potentially infinite sequence of transactions) and for updating the results incrementally after each new transaction. Stream mining algorithms assume that each transaction in a database can only be read once. The estDec+ algorithm is also interesting because it mines recent frequent itemsets, which means that it put more weight on recent transactions than on older transactions when searching for recent frequent itemsets. This allows estDec+ to learn new trends and to forgot older trends.

What is the input of estDec+?

The input of estDec+ is a stream of transactions and a support threshold minsup. Each transaction is a set of items (symbols). For example, consider the following six transactions (t1, t2, ..., t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction. estDec+ is an algorithm for processing a stream. This means that estDec+ is allowed to read each transaction only once because a stream is assumed to be potentially infinite and coming at high speed.

Transaction ID Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 5}

What is the output of estDec+?

estDec+ produces as output the set of recent frequent itemsets contained in the transactions that estDec+ has seen until now. It is said that estDec mines recent frequent itemsets because estDec utilizes a decay function so that estDec puts more weight on recent transactions than on older ones (frequency of an itemset). This allows estDec+ to learn new trends and to forgot older trends.

The output is a set of recent frequent itemsets. The support count of an itemset is the number of transactions that contains the itemset. For example, the itemset {1, 2, 4} has a support count of 1 because it only appear in t1. The support of an itemset is the number of transaction were the itemset appears divided by the total number of transactions seen until now. A frequent itemset is an itemset that has a support higher or equal to minsup.

The estDec+ algorithm is an approximate algorithm. It approximate the support of itemsets and returns itemsets that have an estimated support higher or equal to minsup.

For example, consider the example MainTestEstDecPlus_saveToFile.java. This example consists of loading the transactions from a file named "contextIGB.txt" provided in the SPMF distribution. Then, this example show how to save the result to a file. Here is the output:

2 5 #SUP: 1.0
1 4 5 #SUP: 0.5
1 2 3 #SUP: 0.5
5 #SUP: 1.0
1 2 5 #SUP: 0.5
1 #SUP:0.66
1 5 #SUP: 0.5555555555555556
1 2 4 #SUP: 0.5
4 5 #SUP: 0.5
2 4 #SUP: 0.5
1 4 #SUP: 0.5555555555555556
1 3 #SUP: 0.5555555555555556
4 #SUP: 0.5
1 3 5 #SUP: 0.5
2 3 #SUP:0.66
1 2 #SUP: 0.5555555555555556
3 4 #SUP:0.66
2 #SUP: 1.0
3 5 #SUP:0.66
2 4 5 #SUP: 0.5
3 #SUP:0.66
2 3 5 #SUP:0.66

For example, consider line 1. It indicates that the pattern {1, 2, 5} is a recent frequent itemsets with an estimated support of 50%

Note that we also provide a second example named MainTestEstDec_saveToMemory.java. This example shows how to process a set of transactions from memory instead of from a file and to keep the result into memory instead of saving the result to a file. This is especially useful, if you wish to integrate estDec into another Java program. The example also shows how to set the decay rate.

Input file format

The estDec algorithm can either take as input a stream in memory or read transactions from a file. The input file format of estDec is defined as follows. It is a text file. An item is represented by a positive integer. A transaction is a line in the text file. In each line (transaction), items are separated by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line.

For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 5

Output file format

The output file format of estDec+ is defined as follows. It is a text file, where each line represents a recently frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it is followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by an integer indicating the support of the itemset, expressed as a number of transactions. For example, here are few lines of the output file for this example.

1 2 3 #SUP: 0.5
5 #SUP: 1.0
1 2 5 #SUP: 0.5

The output file here consists of the first line indicates that the itemset {1, 2, 3} has an estimated support of 50 %.

Performance

estDec+ is a reasonably efficient algorithm. When minsup is high, it may use less memory than the original estDec algorithm because the CP-Tree is generally smaller than the estTree.

Where can I get more information about this algorithm?

The estDec+ algorithm is described in this paper:

Se Jung Shin , Dae Su Lee , Won Suk Lee, “CP-tree: An adaptive synopsis structure for compressing frequent itemsets over online data streams”, Information Sciences,Volume 278, 10 September 2014, Pages 559–576

Example 26 : Mining Frequent Itemsets from Uncertain Data Using the U-Apriori Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "UApriori" algorithm, (2) select the input file "contextUncertain.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum expected support to 0.10 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run UApriori contextUncertain.txt output.txt 10% in a folder containing spmf.jar and the example input file contextUncertain.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestUApriori.java" in the package ca.pfv.SPMF.tests.

What is UApriori?

UApriori is an algorithm for mining frequent itemsets from a transaction database where the data is uncertain (contains probabilities). The UApriori algorithm was proposed by Chui et al. (2007).

This algorithm can have multiple applications such as in mining medical data or sensor data where observations may be uncertain.

What is the input ?

UApriori takes as input a transaction database containing probabilities and a minimum expected support threshold (a value between 0 and 1). A transaction database is a set of transactions where each transaction is a set of items. In UApriori, we assume that each item in a transaction is annotated with an existential probability. For example, let's consider the following transaction database, consisting of 4 transactions (t1,t2...t5) and 5 items (1,2,3,4,5). The transaction t1 contains item 1 with a probability of 0.5, item 2 with a probability of 0.4, item 4 with a probability of 0.3 and item 5 with a probability of 0.7. This database is provided in the file "contextUncertain.txt" of the SPMF distribution:


1 2 3 4 5
t1 0.5 0.4
0.3 0.7
t2
0.5 0.4
0.4
t3 0.6 0.5
0.1 0.5
t4 0.7 0.4 0.3
0.9

What is the output?

The output of U-Apriori is the set of frequent itemsets. Note that the definition of a frequent itemset is here different from the definition used by the regular Apriori algorithm because we have to consider the existential probabilities.

The expected support of an itemset in a transaction is defined as the product of the existential probability of each item from the itemset in this transaction. It is a value between 0 and 1. For example, the support of itemset {1, 2} in transaction t1 is 0.5 x 0.4 = 0.2. The expected support of an itemset in a transaction database is the sum of its support in all transactions where it occurs. For example, the expected support of itemset {2, 3} is the sum of its expected support in t2 and t4 : 0.5 x 0.4 + 0.4 x 0.3 = 0.32. A frequent itemset is an itemset that has an expected support higher or equal to the minimum expected support set by the user. For example, by running U-Apriori with a minimum expected support of 0.10, we obtain 19 frequent itemsets, including:

itemsets expected support
{2 3 5} 0.19
{1 3 5} 0.19
{1 4 5} 0.14
{2 4 5} 0.11
{1 2 5} 0.54
{1 5} 1.28
{1 3} 0.21
{1 4} 0.21
{2 3} 0.32
{1 2} 0.78
... ...

Input file format

The input file format of UApriori is defined as follows. It is a text file. An item is represented by a positive integer. Each item is associated with a probability indicated as a double value between parenthesis. A transaction is a line in the text file. In each line (transaction), each item is immediately followed by its probability between parenthesis and a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same line. Probabilities should be greater than 0 and not more than 1.

For example, for the previous example, the input file is defined as follows:

# This binary context contains uncertain data.
# Each line represents a transaction.
# For each item there is an existential probability.
1(0.5) 2(0.4) 4(0.3) 5(0.7)
2(0.5) 3(0.4) 5(0.4)
1(0.6) 2(0.5) 4(0.1) 5(0.5)
1(0.7) 2(0.4) 3(0.3) 5(0.9)

The first line represents the itemsets {1, 2, 4, 5} where items 1, 2, 4 and 5 respectively have the probabilities 0.5, 0.4, 0.3 and 0.7.

Output file format

The output file format of UApriori is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer and it expected support between parenthesis, followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by a double value indicating the expected support of the itemset. For example, we show below the output file for this example.

2 (0.4) Support: 1.7999999999999998
3 (0.4) Support: 0.7
4 (0.3) Support: 0.4
5 (0.7) Support: 2.5
1 (0.5) Support: 1.8
1 (0.5) 2 (0.4) Support: 0.78
1 (0.5) 3 (0.4) Support: 0.21
2 (0.4) 5 (0.7) Support: 1.09
2 (0.4) 4 (0.3) Support: 0.16999999999999998
1 (0.5) 5 (0.7) Support: 1.2799999999999998
3 (0.4) 5 (0.7) Support: 0.43000000000000005
2 (0.4) 3 (0.4) Support: 0.32
1 (0.5) 4 (0.3) Support: 0.21
4 (0.3) 5 (0.7) Support: 0.26
1 (0.5) 3 (0.4) 5 (0.7) Support: 0.189
2 (0.4) 3 (0.4) 5 (0.7) Support: 0.188
1 (0.5) 2 (0.4) 5 (0.7) Support: 0.542
1 (0.5) 4 (0.3) 5 (0.7) Support: 0.135
2 (0.4) 4 (0.3) 5 (0.7) Support: 0.10899999999999999

For example, the last line indicates that the itemset {2, 4, 5} has an expected support of 0.1089999 and that items in this itemset have an existential support of 0.4, 0.3 and 0.7 with respect to this itemset, respectively.

Performance

UApriori is not the most efficient algorithm for uncertain itemset mining but it is simple and it is the first algorithm designed for this task.

Where can I get more information about the UApriori algorithm?

Here is an article describing the UApriori algorithm:

C. Kit Chui, B. Kao, E. Hung: Mining Frequent Itemsets from Uncertain Data. PAKDD 2007: 47-58

Example 27 : Mining Erasable Itemsets from a Product Database with the VME algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "VME" algorithm, (2) select the input file "contextVME.txt", (3) set the output file name (e.g. "output.txt") (4) set the threshold to 15%. and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run VME contextVME.txt output.txt 15% in a folder containing spmf.jar and the example input file contextVME.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestVME.java" in the package ca.pfv.SPMF.tests.

What is the VME algorithm?

VME (Deng & Xu, 2010) is an algorithm for mining erasable itemsets from a product database with profit information.

What is the input?

VME takes as input a product database and a threshold (a value between 0 and 100%). A product is defined as a set of items that are used to assemble the product. Moreover each product is annotated with a profit (a positive integer) that indicates how much money this product generate for the company. For example, let's consider the following product database, consisting of 6 products and 7 items (this example is taken from the article of Deng & Xu, 2010). Each product is annotated with the profit information. For example, the first line indicates that the product 1 generate a total profit of 50 $ for the company and that its assembly requires parts 2, 3, 4 and 6. This product database is provided in the file "contextVME.txt" of the SPMF distribution.:


profit items
product1 50$ {2, 3, 4, 6}
product2 20$ {2, 5, 7}
product3 50$ {1, 2, 3, 5}
product4 800$ {1, 2, 4}
product5 30$ {6, 7}
product6 50$ {3, 4}

What is the output?

The output is the set of erasable itemsets generating a loss of profit lower or equal to the user-specificed threshold. The idea is to discover item that the company could stop manufacturing and that would minimize the amount of profit lost by being unable to build products.

To explain what is an erasable itemset more formally, it is necessary to review some definitions An itemset is an unordered set of distinct items. The loss of profit generated by an itemset is defined as the sum of the product profit for all products containing an item from this itemset. For example, the lost of profit of itemset {5, 6} is the sum of the profits of products containing 5 and/or 6: 50$ + 20 $ + 50 $ + 30 $ = 150 $. The loss of profit can also be expressed as a percentage of the total profit of the database. For example, in this database the total profit is 50 + 20 + 50 + 800 + 30 + 50 = 1000$. Therefore, the lost of profit by the itemset {5, 6} could be expressed as 15% (150 / 1000 * 100).

By running VME with a threshold of 15 %, we obtain 8 erasable itemsets (having a profit loss less or equal to 15% x 1000$ = 150 $):

erasable itemsets loss of profit ("gain")
{3} 150
{5} 70
{6} 80
{7} 50
{5 6} 150
{5 7} 100
{6 7} 100
{5 6 7} 150

This means that if the items from one of those erasable itemsets are not manufactured anymore, then the loss of profit will be lower or equal to 15%.

Input file format

The input file format of VME is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of two sections, as follows.

  • First, the profit of the transaction is indicated by an integer number, followed by a single space.
  • Second, the items in the transaction are listed. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction.

For example, for the previous example, the input file is defined as follows:

50 2 3 4 6
20 2 5 7
50 1 2 3 5
800 1 2 4
30 6 7
50 3 4

Consider the first line. It means that the transaction {2, 3, 4, 6} has a profit of 50 and it contains the items 2, 3, 4 and 6. The following lines follow the same format.

Output file format

The output file format of VME is defined as follows. It is a text file, where each line represents an erasable itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#LOSS:" appears, which is followed by a integer value indicating the loss of profit for that itemset.

3 #LOSS: 150
5 #LOSS: 70
6 #LOSS: 80
7 #LOSS: 50
5 6 #LOSS: 150
5 7 #LOSS: 100
6 7 #LOSS: 100
5 6 7 #LOSS: 150

For example, the first line indicates that the itemset {3} would generate a loss of profit of 150. The following lines follows the same format.

Performance

The VME algorithm is Apriori-based. It is not the fastest algorithm for this problem. But it is the only one available in SPMF because this problem is not very popular. For more efficient algorithms for this problem, you can search for the author names. They have proposed a few algorithms with some improvements.

Where can I get more information about the VME algorithm?

Here is an article describing the VME algorithm:

Z. Deng, X. Xu: An Efficient Algorithm for Mining Erasable Itemsets. ADMA (1) 2010: 214-225.

Example 28 : Building, updating incrementally and using an Itemset-Tree to generate targeted frequent itemsets and association rules.

How to run this example?

  • This algorithm is not offered in the release version of SPMF.
  • To run this example with the source code version of SPMF, launch the file "MainTestItemsetTree.java" in the package ca.pfv.SPMF.tests.

What is an itemset-tree?

An itemset-tree is a special data structure that can be used for performing efficient queries about itemsets and association rules in a transaction database without having to generate all of them beforehand.

An itemset-tree has the nice property of being incremental, which means that new transactions can be added to an existing itemset tree very efficiently without having to rebuild the tree from scratch. An itemset-tree also has the property of being compact.

How to use it?

An itemset-tree is built by inserting a set of transactions into the tree. A transaction is simply a set of distinct items. For example, we could insert the following 6 transactions (t1,t2...t5) into an itemset-tree. In this example, the transaction t1 represents the set of items {1, 4}. This set of transactions is provided in the file "contextItemsetTree.txt" of the SPMF distribution.

transaction IDs items
t1 {1, 4}
t2 {2, 5}
t3 {1, 2, 3, 4, 5}
t4 {1, 2, 4}
t5 {2, 5}
t6 {2, 4}

The result of the insertion of these six transactions is the following itemset-tree (see the article by Kubat for more details).

{}   sup=6
[2 ] sup=3
[2 5 ] sup=2
[2 4 ] sup=1
[1 ] sup=3
[1 2 ] sup=2
[1 2 4 ] sup=1
[1 2 3 4 5 ] sup=1
[1 4 ] sup=1

The root is the empty itemset {} and the leafs are {2, 5}, {2, 4}, {1 2 4},{1 2 3 4 5} and {1, 4}.

Once an itemset-tree has been created, it is possible to update it by inserting a new transaction. For example, in this example provided in the source code, we update the previous tree by adding a new transaction {4, 5}. The result is this tree:

{}   sup=7
[2 ] sup=3
[2 5 ] sup=2
[2 4 ] sup=1
[1 ] sup=3
[1 2 ] sup=2
[1 2 4 ] sup=1
[1 2 3 4 5 ] sup=1
[1 4 ] sup=1
[4 5 ] sup=1

Next, it is shown how to query the tree to determine the support of a target itemset efficiently. For example, if we execute the query of finding the support of the itemset {2}, the support is determined to be 5 because 2 appear in 5 transactions.

After that the source code offers an example of how to use the itemset tree to get all itemsets that subsume an itemset and to get their support. For example, if we use the itemset {1 2} for this query the result is:

[1 2 ]    supp:2
[1 2 3 ] supp:1
[1 2 4 ] supp:2
[1 2 5 ] supp:1
[1 2 3 4 ] supp:1
[1 2 3 5 ] supp:1
[1 2 4 5 ] supp:1
[1 2 3 4 5 ] supp:1

Another example provided is how to use the tree to find all itemsets that subsume an itemset such that the support is higher or equal to a user-specified threshold named minsup (a positive integer representing a number of transactions). For example, if we execute this query with the itemset {1} and minsup =2, we get this result:

[1 ]    supp:3
[1 2 ] supp:2
[1 4 ] supp:3
[1 2 4 ] supp:2

Lastly, another example is how to generate all association rules having a target itemset as antecedent and a support and confidence respectively higher or equal to some user-specificed thresholds minsup (a positive integer representing a number of transactions) and minconf (a value between 0 and 1). For example, if the target itemset is {1} and minconf = 0.1 and minsup = 2, the result is:

[ 1  ] ==> [2  ]  sup=2  conf=0.666666666666666

[ 1 ] ==> [4 ] sup=3 conf=1.0

[ 1 ] ==> [2 4 ] sup=2 conf=0.66666666666666

Input and output file format

There is no need to use an input and output file with an itemset tree because it is an incremental data structure that is designed for live update and live targeted queries rather than batch processing.

However, it is possible to load a transaction database in an itemset tree. In this case, a file is loaded. The file is defined as a text file where each line represents a transactions. Each item is represented by an integer and it is assumed that all transactions are sorted according to a total order and that no item can appear twice in the same transaction. On any given line, the items of the corresponding transaction are listed such that each item is separated from the following item by a single space. For example, the file "contextItemsetTree.txt" that is provided contains the following content:

1 4
2 5
1 2 3 4 5
1 2 4
2 5
2 4

There is a total of six transactions (six lines) in the file. The first line represents the transaction {1, 4} (containing items 1 and 4). The second line represents the transaction {2, 5}. The third line represents the transaction {1, 2, 3, 4, 5}. The following lines follow the same format.

Performance

The itemset-tree is an efficient data structure for the case of a database that needs to be updated frequently and where targeted queries need to be performed. For details about the complexity in terms of space and time, please refer to the article by Kubat et al., which provides an extensive discussion of the complexity

Where can I get more information about the Itemset-tree data structure and related algorithms?

This article describes the itemset-tree and related algorithms for querying it:

Miroslav Kubat, Aladdin Hafez, Vijay V. Raghavan, Jayakrishna R. Lekkala, Wei Kian Chen: Itemset Trees for Targeted Association Querying. IEEE Trans. Knowl. Data Eng. 15(6): 1522-1534 (2003)

Example 29 : Building, updating incrementally and using a Memory Efficient Itemset-Tree to generate targeted frequent itemsets and association rules.

How to run this example?

  • This algorithm is not offered in the release version of SPMF.
  • To run this example with the source code version of SPMF, launch the file "MainTestMemoryEfficientItemsetTree.java" in the package ca.pfv.SPMF.tests.

What is a Memory-Efficient Itemset-Tree (MEIT)?

An itemset-tree (IT) is a special data structure that can be used for performing efficient queries about itemsets and association rules in a transaction database without having to generate all of them beforehand.

An itemset-tree has the nice property of being incremental, which means that new transactions can be added to an existing itemset tree very efficiently without having to rebuild the tree from scratch. An itemset-tree also has the property of being compact.

The Memory-Efficient Itemset-Tree (MEIT) is a modification of the original Itemset-Tree structure that uses about twice less memory than the regular itemset-tree (see the paper describing MEIT for a performance comparison). But it runs about twice slower. Therefore, choosing between using an IT or MEIT is a trade-off between memory and speed.

How to use it?

A Memory-Efficient Itemset-Tree (MEIT) is built by inserting a set of transactions into the tree. A transaction is simply a set of distinct items. For example, we could insert the following 6 transactions (t1,t2...t5) into an itemset-tree. In this example, the transaction t1 represents the set of items {1, 4}. This set of transactions is provided in the file "contextItemsetTree.txt" of the SPMF distribution.

transaction IDs items
t1 {1, 4}
t2 {2, 5}
t3 {1, 2, 3, 4, 5}
t4 {1, 2, 4}
t5 {2, 5}
t6 {2, 4}

The result of the insertion of these six transactions is the following MEIT.

{}   sup=6
[2 ] sup=3
[5 ] sup=2
[4 ] sup=1
[1 ] sup=3
[2 ] sup=2
[4 ] sup=1
[3 5 ] sup=1
[4 ] sup=1

The root is the empty itemset {} and the leafs are {5}, {4}, {4},{3 5} and {4}.

Once an itemset-tree has been created, it is possible to update it by inserting a new transaction. For example, in this example provided in the source code, we update the previous tree by adding a new transaction {4, 5}. The result is this tree:

{}   sup=7
[2 ] sup=3
[5 ] sup=2
[4 ] sup=1
[1 ] sup=3
[2 ] sup=2
[4 ] sup=1
[3 5 ] sup=1
[4 ] sup=1
[4 5 ] sup=1

Next, it is shown how to query the tree to determine the support of a target itemset efficiently. For example, if we execute the query of finding the support of the itemset {2}, the support is determined to be 5 because 2 appear in 5 transactions.

After that the source code offers an example of how to use the itemset tree to get all itemsets that subsume an itemset and to get their support. For example, if we use the itemset {1 2} for this query the result is:

[1 2 ]    supp:2
[1 2 3 ] supp:1
[1 2 4 ] supp:2
[1 2 5 ] supp:1
[1 2 3 4 ] supp:1
[1 2 3 5 ] supp:1
[1 2 4 5 ] supp:1
[1 2 3 4 5 ] supp:1

Another example provided is how to use the tree to find all itemsets that subsume an itemset such that the support is higher or equal to a user-specified threshold named minsup (a positive integer representing a number of transactions). For example, if we execute this query with the itemset {1} and minsup =2, we get this result:

[1 ]    supp:3
[1 2 ] supp:2
[1 4 ] supp:3
[1 2 4 ] supp:2

Lastly, another example is how to generate all association rules having a target itemset as antecedent and a support and confidence respectively higher or equal to some user-specificed thresholds minsup (a positive integer representing a number of transactions) and minconf (a value between 0 and 1). For example, if the target itemset is {1} and minconf = 0.1 and minsup = 2, the result is:

[ 1  ] ==> [2  ]  sup=2  conf=0.666666666666666

[ 1 ] ==> [4 ] sup=3 conf=1.0

[ 1 ] ==> [2 4 ] sup=2 conf=0.66666666666666

Input and output file format

There is no need to use an input and output file with aa memory-efficient itemset tree because it is an incremental data structure that is designed for live update and live targeted queries rather than batch processing.

However, it is possible to load a transaction database in a memory-efficient itemset tree. In this case, a file is loaded. The file is defined as a text file where each line represents a transactions. Each item is represented by an integer and it is assumed that all transactions are sorted according to a total order and that no item can appear twice in the same transaction. On any given line, the items of the corresponding transaction are listed such that each item is separated from the following item by a single space. For example, the file "contextItemsetTree.txt" that is provided contains the following content:

1 4
2 5
1 2 3 4 5
1 2 4
2 5
2 4

There is a total of six transactions (six lines) in the file. The first line represents the transaction {1, 4} (containing items 1 and 4). The second line represents the transaction {2, 5}. The third line represents the transaction {1, 2, 3, 4, 5}. The following lines follow the same format.

Performance

The Memory-Efficient Itemset-Tree (MEIT) is an efficient data structure for the case of a database that needs to be updated frequently and where targeted queries need to be performed on itemsets and association rules.

The MEIT is a modification of the original Itemset-Tree (MEIT). According to our experiments, the MEIT uses about twice less memory than the IT but is about twice slower for answering queries. Therefore, choosing between MEIT and IT is a compromise between speed and memory.

Where can I get more information about the Itemset-tree data structure and related algorithms?

This article describes the Memory-Efficient Itemset-tree:

Fournier-Viger, P., Mwamikazi, E., Gueniche, T., Faghihi, U. (2013). Memory Efficient Itemset Tree for Targeted Association Rule Mining. Proc. 9th International Conference on Advanced Data Mining and Applications (ADMA 2013) Part II, Springer LNAI 8347, pp. 95-106.

Example 30 : Mining Frequent Itemsets with Multiple Support Thresholds Using the MSApriori Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "MSApriori" algorithm, (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set beta = 0.4 and LS = 0.2 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run MSApriori contextIGB.txt output.txt 0.4 0.2 in a folder containing spmf.jar and the example input file contextIGB.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestMSApriori_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is MISApriori?

MISApriori is an algorithm for mining frequent itemsets by using multiple minimum supports. It is a generalization of the Apriori algorithm, which uses a single minimum support threshold.

The idea behind MSApriori is that different minimum supports could be used to consider the fact that some items are less frequent than others in a dataset.

What is the input of this algorithm?

The input of MSApriori is a transaction database and two parameters named beta (a value between 0 and 1) and LS (a value between 0 and 1). These parameters are used to determine a minimum support for each item.

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

What is the output of this algorithm?

The output of MSApriori is the set of all frequent itemsets contained in the database.

Contrarily to the original Apriori algorithm, MSApriori use multiple minimum supports thresholds instead of just one. In fact, MSApriori uses a minimum support value for each item. Because it would be time consuming to set a minimum support threshold value for each item for a large database, the thresholds are determined automatically by using two user-specified parameters named beta (0 <= B <= 1) and LS (0 <= LS <= 1).

The minimum support of an item k is then defined as the greatest value between:

  • LS
  • and B x f(k) where f(k) is the number of transactions containing the item k.

Note that if B is set to 0, there will be a single minimum support for all items and this will be equivalent to the regular Apriori algorithm.

The support of an itemset is the number of transactions containing the itemset divided by the total number of transactions. An itemset is a frequent itemset if its support is higher or equal to the smallest minimum support threshold from the minimum support thresholds of all its items.

Why MSApriori is useful? It is useful because it allows discovering frequent itemsets containing rare items (if their minimum support is set low).

If we run MSApriori on the previous transaction database with beta = 0.4 and LS = 0.2, we obtain the following result:

1 supp: 4
2 supp: 6
3 supp: 4
4 supp: 4
5 supp: 5
1 2 Support: 4
1 3 Support: 2
1 4 Support: 3
1 5 Support: 4
2 3 Support: 4
2 4 Support: 4
2 5 Support: 5
3 4 Support: 2
3 5 Support: 3
4 5 Support: 3
1 2 3 Support: 2
1 2 4 Support: 3
1 2 5 Support: 4
1 3 5 Support: 2
1 4 5 Support: 3
2 3 4 Support: 2
2 3 5 Support: 3
2 4 5 Support: 3
1 2 3 5 Support: 2
1 2 4 5 Support:
Note that here the support is expressed by an integer value which represents the number of transactions containing the itemset. For example, itemset {2, 3 5} has a support of 3 because it appears in three transactions, namely t2, t4 and t5. This integer value can be converted as a percentage by dividing by the total number of transactions.

Input file format

The input file format of MSApriori is defined as follows. It is a text file. Each lines represents a transaction. The items in the transaction are listed. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

Consider the first line. It means that the first transaction is the itemset {1, 2, 4, 5}. The following lines follow the same format.

Output file format

The output file format of MSApriori is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by a integer value indicating the support of that itemset.

1 #SUP: 4
2 #SUP: 6
3 #SUP: 4
4 #SUP: 4
5 #SUP: 5
1 2 #SUP: 4
1 3 #SUP: 2
1 4 #SUP: 3
1 5 #SUP: 4
2 3 #SUP: 4
2 4 #SUP: 4
2 5 #SUP: 5
3 4 #SUP: 2
3 5 #SUP: 3
4 5 #SUP: 3
1 2 3 #SUP: 2
1 2 4 #SUP: 3
1 2 5 #SUP: 4
1 3 5 #SUP: 2
1 4 5 #SUP: 3
2 3 4 #SUP: 2
2 3 5 #SUP: 3
2 4 5 #SUP: 3
1 2 3 5 #SUP: 2
1 2 4 5 #SUP: 3

For example, the first line indicates that the itemset {1} has a support of 4 transactions. The following lines follows the same format.

Performance

MSApriori is one of the first algorithm for mining itemsets with multiple minimum support thresholds. It is not the most efficient algorithm for this task because it is based on Apriori and thus suffer from the same limitations. If performance is important, it is recommend to use CFPGrowth++, which is based on FPGrowth and is more efficient.

Note that there is one important difference between the input of CFPGrowth++ and MSApriori in SPMF. The MISApriori works by setting the multiple minimum supports by using the LS and BETA values. The CFPGrowth++ implementation uses a list of minimum support values stored in a text file instead.

Where can I get more information about the MSApriori algorithm?

This article describes the MSApriori algorithm:

B. Liu, W. Hsu, Y. Ma, "Mining Association Rules with Multiple Minimum Supports" Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-99), August 15-18, 1999, San Diego, CA, USA.

Example 31 : Mining Frequent Itemsets with Multiple Support Thresholds Using the CFPGrowth++ Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "CFPGrowth++" algorithm, (2) select the input file "contextCFPGrowth.txt", (3) set the output file name (e.g. "output.txt") (4) put "MIS.TXT" in the "MIS file name" text field (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run CFPGrowth++ contextCFPGrowth.txt output.txt MIS.txt in a folder containing spmf.jar and the example input file contextCFPgrowth.txt and MIS.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestCFPGrowth_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is CFPGrowth++?

CFPGrowth++ is an algorithm for mining frequent itemsets by using multiple minimum supports. It is an extension of the CFPGrowth algorithm for mining frequent itemsets using multiple minimum support thresholds.

What is the input of this algorithm?

The input of CFPGrowth++ is a transaction database and a list of minimum support thresholds indicating the minimum support threshold for each item.

A transaction database is a set of transactions, where each transaction is a list of distinct items (symbols). For example, let's consider the following transaction database. It consists of 5 transactions (t1,t2...t6) and 8 items (1,2,3,4,5,6,7,8). For instance, transaction t1 is the set of items {1, 3, 4, 6}. This database is provided in the file "contextCFPGrowth.txt" of the SPMF distribution.. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction ID items
t1 {1, 3, 4,6}
t2 {1, 3, 5, 6, 7}
t3 {1, 2, 3, 6, 8}
t4 {2, 6, 7}
t5 {2, 3}

The list of minimum support threshold is stored in a text file that is read as input by the algorithm. This is provided in the file "MIS.txt":

item minimum support threshold
1 1
2 2
3 3
4 3
5 2
6 3
7 2
8 1

This file indicated for example that the minimum support threshold to be used for item 6 is 3.

What is the output of this algorithm?

The output of CFPgrowth++ is the set of all frequent itemsets contained in the database.

What is a frequent itemset ? The support of an itemset is the number of transactions containing the itemset. An itemset is a frequent itemset if its support is higher or equal to the smallest minimum support threshold among the minimum support thresholds of all its items. For example, the itemset {1 2 8} is frequent because it appears in one transactions (t3) and its support is higher than the smallest minimum support among the minimum support of item 1, item 2 and item 8, which are respectively 1, 2 and 1.

Why CFPGrowth++ is useful? It is useful because it permits setting lower minimum support thresholds for rare items. Therefore, it allows discovering frequent itemsets containing rare items.

If we run CFPGrowth++ on the previous transaction database with the MIS.txt file previously described, we get the following result, where each line represents an itemsets followed by ":" and then its absolute support.:

8:1
8 1:1
8 1 2:1 // for example, this itemset is {1, 2, 8}, and it has a support of 1.
8 1 2 6:1
8 1 2 6 3:1
8 1 2 3:1
8 1 6:1
8 1 6 3:1
8 1 3:1
8 2:1
8 2 6:1
8 2 6 3:1
8 2 3:1
8 6:1
8 6 3:1
8 3:1
1:3 // for example, this itemset is {1}, and it has a support of 3.
1 7:1
1 7 5:1
1 7 5 6:1
1 7 5 6 3:1
1 7 5 3:1
1 7 6:1
1 7 6 3:1
1 7 3:1
1 5:1
1 5 6:1
1 5 6 3:1
1 5 3:1
1 2:1
1 2 6:1
1 2 6 3:1
1 2 3:1
1 6:3
1 6 4:1
1 6 4 3:1
1 6 3:3
1 4:1
1 4 3:1
1 3:3
7:2
7 6:2
2:3
2 6:2
2 3:2
6:4
6 3:3
3:4
Note: If you are using the GUI version of SPMF the file containing the minimum support must be located in the same folder as the input file containing the transaction database.

Input file format

The input file format of CFPGrowth++ is two files defined as follows.

The first file (e.g. contextCFPGrowth.txt) It is a text file containing the transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 3 4 6
1 3 5 6 7
1 2 3 6 8
2 6 7
2 3

Consider the first line. It means that the first transaction is the itemset {1, 3, 4, 6}. The following lines follow the same format.

The second file is a text file (e.g. MIS.txt) which provides the minimum support to be used for each item. Each line indicate the minimum support for an item and consists of two integer values separated by a single space. The first value is the item. The second value is the minimum support value to be used for this item. For example, here is the file used in this example. The first line indicate that for item "1" the minimum support to be used is 1 (one transaction). The other lines follow the same format.

1 1
2 2
3 3
4 3
5 2
6 3
7 2
8 1

Output file format

The output file format of CFPGrowth++ is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by a integer value indicating the support of that itemset.

8 #SUP: 1
8 1 #SUP: 1
8 1 2 #SUP: 1
8 1 2 6 #SUP: 1
8 1 2 6 3 #SUP: 1
8 1 2 3 #SUP: 1
8 1 6 #SUP: 1
8 1 6 3 #SUP: 1
8 1 3 #SUP: 1
8 2 #SUP: 1
8 2 6 #SUP: 1
8 2 6 3 #SUP: 1
8 2 3 #SUP: 1
8 6 #SUP: 1
8 6 3 #SUP: 1
8 3 #SUP: 1
1 #SUP: 3
1 7 #SUP: 1
1 7 5 #SUP: 1
1 7 5 6 #SUP: 1
1 7 5 6 3 #SUP: 1
1 7 5 3 #SUP: 1
1 7 6 #SUP: 1
1 7 6 3 #SUP: 1
1 7 3 #SUP: 1
1 5 #SUP: 1
1 5 6 #SUP: 1
1 5 6 3 #SUP: 1
1 5 3 #SUP: 1
1 2 #SUP: 1
1 2 6 #SUP: 1
1 2 6 3 #SUP: 1
1 2 3 #SUP: 1
1 6 #SUP: 3
1 6 4 #SUP: 1
1 6 4 3 #SUP: 1
1 6 3 #SUP: 3
1 4 #SUP: 1
1 4 3 #SUP: 1
1 3 #SUP: 3
7 #SUP: 2
7 6 #SUP: 2
2 #SUP: 3
2 6 #SUP: 2
2 3 #SUP: 2
6 #SUP: 4
6 3 #SUP: 3
3 #SUP: 4

For example, the last line indicates that the itemset {4} has a support of 4 transactions. The other lines follows the same format.

Implementation details

In the source code version of SPMF, there are two versions of CFPGrowth: one that saves the result to a file (MainTestCFPGrowth_saveToFile.java) and one that saves the result to memory (MainTestCFPGrowth_saveToMemory.java). In the graphical interface and command line interface, only the version that saves to file is offered.

Performance

CFPGrowth++ is a very efficient algorithm. It is based on FPGrowth.

SPMF also offers the MISApriori algorithm, which is less efficient than CFPGrowth++. Note that there is one important difference between the input of CFPGrowth++ and MSApriori in SPMF. The MISApriori algorithm works by setting the multiple minimum supports by using some special parameters named LS and BETA (see the example describing MISApriori for more details). The CFPGrowth++ implementation instead uses a list of minimum support values stored in a text file.

Where can I get more information about the CFPGrowth++ algorithm?

This article describes the original CFPGrowth algorithm:

Y.-H. Hu, Y.-L. Chen: Mining association rules with multiple minimum supports: a new mining algorithm and a support tuning mechanism. Decision Support Systems 42(1): 1-24 (2006)

This article describe CFPGrowth++, the extension of CFPGrowth that is implemented in SPMF, which introduce a few additional optimizations.

Kiran, R. U., & Reddy, P. K. (2011). Novel techniques to reduce search space in multiple minimum supports-based frequent pattern mining algorithms. In Proceedings of the 14th International Conference on Extending Database Technology, ACM, pp. 11-20.

Example 32 : Mining High-Utility Itemsets from a Database with Utility Information with the Two-Phase Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Two-Phase" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Two-Phase DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestTwoPhaseAlgorithm_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is Two-Phase?

Two-Phase (Liu et al., 2005) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information.

High utility itemset mining has several applications such as discovering groups of items in transactions of a store that generate the most profit. A database containing utility information is a database where items can have quantities and a unit price. Although these algorithms are often presented in the context of market basket analysis, there exist other applications.

What is the input?

Two-phase takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of Two-Phase is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run Two Phase with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)
{1 2 3 4 5 6} 30 20 % (1 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of Two-phase is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of Two-phase is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by a double value indicating the support of the itemset. Then, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #SUP: 0.4 #UTIL: 30
2 5 #SUP: 0.6 #UTIL: 31
1 3 5 #SUP: 0.4 #UTIL: 31
2 3 4 #SUP: 0.4 #UTIL: 34
2 3 5 #SUP: 0.6 #UTIL: 37
2 4 5 #SUP: 0.4 #UTIL: 36
2 3 4 5 #SUP: 0.4 #UTIL: 40
1 2 3 4 5 6 #SUP: 0.2 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a support of 0.4 and a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a much more difficult problem than frequent itemset mining. Therefore, algorithms for high-utility itemset mining are generally slower than frequent itemset mining algorithms.

The Two-Phase algorithm is an important algorithm because it introduced the concept of mining high utility itemset by using two phases by first overestimating the utility of itemsets in phase I and then calculating their exact utility in phase II. However, there are now some more efficient algorithms. For efficiency, it is recommended to use a more efficient algorithm such as EFIM that is also included in SPMF and is one of the most efficient algorithm for this problem (see performance page of this website).

Implementation details

In the source code version of SPMF, there are two versions of Two-Phase: one that saves the result to a file (MainTestTwoPhaseAlgorithm_saveToFile.java) and one that saves the result to memory (MainTestTwoPhaseAlgorithm_saveToMemory.java). In the graphical interface and command line interface, only the version that saves to file is offered.

Also note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the Two-Phase algorithm?

Here is an article describing the Two-Phase algorithm:

Y. Liu, W.-K. Liao, A. N. Choudhary: A Two-Phase Algorithm for Fast Discovery of High Utility Itemsets. PAKDD 2005: 689-695

Example 33 : Mining High-Utility Itemsets from a Database with Utility Information with the FHM Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FHM" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FHM DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFHM.java" in the package ca.pfv.SPMF.tests.

What is FHM?

FHM (Fournier-Viger et al., ISMIS 2014) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information.

High utility itemset mining has several applications such as discovering groups of items in transactions of a store that generate the most profit. A database containing utility information is a database where items can have quantities and a unit price. Although these algorithms are often presented in the context of market basket analysis, there exist other applications.

What is the input?

FHM takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of FHM is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run FHM with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)
{1 2 3 4 5 6} 30 20 % (1 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of FHM is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of FHM is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms.

The FHM algorithm was shown to be up to six times faster than HUI-Miner (also included in SPMF), especially for sparse datasets (see the performance section of the website for a comparison). But the EFIM algorithm (also included in SPMF) greatly outperforms FHM (see performance section of the website).

Implementation details

The version offered in SPMF is the original implementation of FHM.

Note that the input format is not exactly the same as described in the article. But it is equivalent.

Where can I get more information about the FHM algorithm?

This is the reference of the article describing the FHM algorithm:

Fournier-Viger, P., Wu, C.-W., Zida, S., Tseng, V. (2014) FHM: A Faster High-Utility Itemset Mining Algorithm using Estimated Utility Co-occurrence Pruning. Proc. 21st International Symposium on Methodologies for Intelligent Systems (ISMIS 2014), Springer, LNAI, pp. 83-92

Example 34 : Mining High-Utility Itemsets from a Database with Utility Information with the EFIM Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "EFIM" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run EFIM DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestEFIM_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is EFIM?

EFIM (Zida et al., 2015) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information.

High utility itemset mining has several applications such as discovering groups of items in transactions of a store that generate the most profit. A database containing utility information is a database where items can have quantities and a unit price. Although these algorithms are often presented in the context of market basket analysis, there exist other applications.

What is the input?

EFIM takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of EFIM is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run FHM with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)
{1 2 3 4 5 6} 30 20 % (1 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of EFIM is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of EFIM is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms.

The EFIM algorithm was shown to be up to two orders of magnitude faster than the previous state-of-the-art algorithm FHM, HUI-Miner, d2HUP, UPGrowth+ (also included in SPMF), and consumes up to four times less memory (see the performance section of the website for a comparison).

Implementation details

The implementation offered in SPMF is the original implementation of EFIM.

In the source code version of SPMF, there are two versions of EFIM: one that saves the result to a file (MainTestEFIM_saveToFile.java) and one that saves the result to memory (MainTestEFIM_saveToMemory.java). In the graphical interface and command line interface, only the version that saves to file is offered.

Note that the input format is not exactly the same as described in the article. But it is equivalent.

Where can I get more information about the EFIM algorithm?

This is the reference of the article describing the EFIM algorithm:

Zida, S., Fournier-Viger, P., Lin, J. C.-W., Wu, C.-W., Tseng, V.S. (2015). EFIM: A Highly Efficient Algorithm for High-Utility Itemset Mining. Proceedings of the 14th Mexican Intern. Conference on Artificial Intelligence (MICAI 2015), Springer LNAI, to appear.

Example 35 : Mining High-Utility Itemsets from a Database with Utility Information with the HUI-Miner Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUI-Miner" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUI-Miner DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHUIMiner.java" in the package ca.pfv.SPMF.tests.

What is HUI-Miner?

HUI-Miner (Liu & Qu, CIKM 2012) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information.

High utility itemset mining has several applications such as discovering groups of items in transactions of a store that generate the most profit. A database containing utility information is a database where items can have quantities and a unit price. Although these algorithms are often presented in the context of market basket analysis, there exist other applications.

What is the input?

HUI-Miner takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of HUI-Miner is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run HUI-Miner with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)
{1 2 3 4 5 6} 30 20 % (1 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of HUI-Miner is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of HUI-Miner is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The HUI-Miner algorithm is reported as one of the most efficient algorithm for high utility itemset mining. However, recently the FHM algorithm (also included in SPMF) was shown to be up to six times faster than HUI-Miner, especially for sparse datasets (see the performance section of the website for a comparison). More recently, the EFIM algorithm (2015) was proposed and was shown to outperform FHM (2014), HUI-Miner (2012), HUP-Miner (2014). All these algorithms are offered in SPMF (see "performance" page of this website).

Implementation details

The version implemented here contains all the optimizations described in the paper proposing HUI-Miner. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the HUI-Miner algorithm?

This is the reference of the article describing the HUI-Miner algorithm:

M. Liu, J.-F. Qu: Mining high utility itemsets without candidate generation. CIKM 2012, 55-64

Example 36 : Mining High-Utility Itemsets from a Database with Utility Information with the HUP-Miner Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUP-Miner" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUP-Miner DB_utility.txt output.txt 30 2 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHUPMiner.java" in the package ca.pfv.SPMF.tests.

What is HUP-Miner?

HUP-Miner (Krishnamoorthy, 2014) is an extension of the HUI-Miner algorithm (Liu & Qu, CIKM 2012) for discovering high-utility itemsets in a transaction database containing utility information. It introduces the idea of partitioning the database and another pruning strategy named LA-prune. A drawback of HUP-Miner is that the user needs to set an additional parameter, which is the number of partitions. Moreover, according to our experiments, HUP-Miner is faster than HUI-Miner but slower than FHM.

High utility itemset mining has several applications such as discovering groups of items in transactions of a store that generate the most profit. A database containing utility information is a database where items can have quantities and a unit price. Although these algorithms are often presented in the context of market basket analysis, there exist other applications.

What is the input?

HUP-Miner takes as input a transaction database with utility information, a minimum utility threshold min_utility (a positive integer) and a number of partitions k.

Note that the parameter k determines how much partitions HUP-Miner uses internally, which influence the performance of HUP-Miner but has no effect on the output of the algorithm. A typical value for k could be 10. However, the optimal value for k may be found empirically for each dataset.

Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of HUP-Miner is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run HUP-Miner with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)
{1 2 3 4 5 6} 30 20 % (1 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of HUP-Miner is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of HUP-Miner is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The HUI-Miner algorithm was reported as one of the most efficient algorithm for high utility itemset mining. HUP-Miner is an extension of HUI-Miner, just like FHM. These two latter are faster than HUI-Miner. However, HUP-Miner introduce a new parameter which is the number of partitions. In our experiment, FHM is faster than HUP-Miner. More recently, the EFIM algorithm (2015) was proposed and was shown to outperform HUP-Miner, and other recent algorithms such as FHM (2014), HUI-Miner (2012), HUP-Miner (2014). All these algorithms are offered in SPMF (see "performance" page of this website).

Implementation details

The version implemented here contains all the optimizations described in the paper proposing HUP-Miner. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the HUP-Miner algorithm?

This is the reference of the article describing the HUP-Miner algorithm:

Krishnamoorthy, S. (2014). Pruning Strategies for Mining High-Utility Itemsets. Expert Systems with Applications.

Example 37 : Mining High-Utility Itemsets from a Database with Utility Information with the UP-Growth / UPGrowth+ Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "UPGrowth" or "UPGrowth+" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run UPGrowth DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestUPGrowth.java" in the package ca.pfv.SPMF.tests.

What is UPGrowth?

UP-Growth (Tseng et al., KDD 2010) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information. UP-Growth+ (Tseng et al., KDD 2012) is an improved version.

Those two algorithms are important algorithms because they introduce some interesting ideas. However, recently some more efficient algorithms have been proposed such as FHM (2014) and HUI-Miner (2012). These latter algorithms were shown to be more than 100 times faster than UP-Growth+ in some cases, and are also offered in SPMF.

What is the input?

UP-Growth takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of UP-Growth is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run UP-Growth with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)
{1 2 3 4 5 6} 30 20 % (1 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of UP-Growth is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of UP-Growth is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The UP-Growth (2010) algorithm was the fastest algorithm for high-utility itemset mining in 2010. However, more efficient algorithm have been proposed. The HUI-Miner (2012) was shown to be up to 100 times faster than UPGrowth, and more recently the FHM algorithm (2014) was shown to be up to six times faster than HUI-Miner. More recently, the EFIM algorithm (2015) was proposed and was shown to outperform UPGrowth+ and other recent algorithms such as FHM (2014), HUI-Miner (2012), HUP-Miner (2014). All these algorithms are offered in SPMF (see "performance" page of this website).

Implementation details

The version implemented here contains all the optimizations described in the paper proposing UP-Growth (strategies DGU, DGN, DLU and DLN). Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the UP-Growth algorithm?

This is the reference of the article describing the UP-Growth algorithm:

V S. Tseng, C.-W. Wu, B.-E. Shie, P. S. Yu: UP-Growth: an efficient algorithm for high utility itemset mining. KDD 2010: 253-262

V. S. Tseng, B.-E. Shie, C.-W. Wu, and P. S. Yu. Efficient algorithms for mining high utility itemsets from transactional databases. IEEE Transactions on Knowledge and Data Engineering, 2012, doi: 10.1109/TKDE.2012.59.

Example 38 : Mining High-Utility Itemsets from a Database with Utility Information with the IHUP Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "IHUP" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run IHUP DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestIHUP.java" in the package ca.pfv.SPMF.tests.

What is IHUP?

IHUP (Ahmed et al., TKDE 2009) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information.

Note that the original IHUP algorithm is designed to be incremental. In this implementation of IHUP can only be run in batch mode.

Also note that more efficient algorithm have been recently proposed such as FHM (2014) and HUI-Miner (2012). These latter algorithms outperforms IHUP by more than an order of magnitude, and are also offered in SPMF.

What is the input?

IHUP takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of IHUP is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run IHUP with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)
{1 2 3 4 5 6} 30 20 % (1 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of IHUP is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of IHUP is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The IHUP (2009) algorithm was the fastest algorithm for high-utility itemset mining in 2009. However, more efficient algorithm have been recently proposed. UPGrowth (2010) is an improved version of IHUP. The HUI-Miner (2012) algorithm outperforms UPGrowth (2009) by more than an order of magnitude, and more recently the FHM algorithm (2014) was shown to be up to six times faster than HUI-Miner. More recently, the EFIM algorithm (2015) was proposed and was shown to outperform IHUP, and other recent algorithms such as FHM (2014), HUI-Miner (2012), HUP-Miner (2014). All these algorithms are offered in SPMF (see "performance" page of this website).

Implementation details

The version of IHUP implemented here is designed to be run in batch mode rather than as an incremental algorithm. Besides, note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the IHUP algorithm?

This is the reference of the article describing the IHUP algorithm:

C. F. Ahmed, S. K. Tanbeer, B.-S. Jeong, Y.-K. Lee: Efficient Tree Structures for High Utility Pattern Mining in Incremental Databases. IEEE Trans. Knowl. Data Eng. 21(12): 1708-1721 (2009)

Example 39 : Mining High-Utility Itemsets from a Database with Utility Information with the d2HUP Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "D2HUP" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run D2HUP DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestD2HUP.java" in the package ca.pfv.SPMF.tests.

What is d2HUP?

d2HUP (Liu et al., ICMD 2012) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information.

It is an algorithm that was shown to be more efficient than UPGrowth and Two-Phase. But in the paper describing d2HUP, the performance was not compared with some recent algorithms such as FHM (2014), HUI-Miner (2012), HUP-Miner (2014).

What is the input?

d2HUP takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of d2HUP is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run d2HUP with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility
{2 4} 30
{2 5} 31
{1 3 5} 31
{2 3 4} 34
{2 3 5} 37
{2 4 5} 36
{2 3 4 5} 40
{1 2 3 4 5 6} 30

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of d2HUP is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of d2HUP is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The d2HUP (2012) algorithm was proposed in 2012 to discover high-utility itemsets without maintaining candidates. A similar idea to avoid candidates was proposed in HUI-Miner (2012) at about the same time. In this implementation, we have implemented d2HUP with all the proposed optimizations. In the paper describing d2HUP, this latter was shown to be more efficient than UPGrowth and Two-Phase. Recently, the EFIM algorithm was proposed (also offered in SPMF). EFIM is shown to outperform d2HUP, and other recent algorithms such as FHM (2014), HUI-Miner (2012), HUP-Miner (2014).

Implementation details

Note that the input format is not exactly the same as described in the original article. But it is equivalent.

We have implemented the CAUL structure using pseudo-projections as suggested in the paper. Also, the

Where can I get more information about the d2HUP algorithm?

This is the reference of the article describing the d2HUP algorithm:

Liu, J., Wang, K., Fung, B. (2012). Direct discovery of high utility itemsets without candidate generation. Proceedings of the 2012 IEEE 12th International Conference on Data Mining. IEEE Computer Society, 2012.

Example 40 : Mining High-Utility Itemsets from a Transaction Database with Utility Information while considering Length Constraints, using the FHM+ algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FHM+" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30, (5) set the minimum length to 2, (6) set the maximum length to 3, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FHM+ DB_utility.txt output.txt 30 2 3 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFHMPlus.java" in the package ca.pfv.SPMF.tests.

What is FHM+?

FHM+ (Fournier-Viger et al., IEA AIE 2016) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information. It extends the FHM algorithm by letting the user specify length constraints to find only patterns having a minimum and maximum size (length), and use novel optimizations to mine patterns with these constraints efficiently. Using constraints on the length of itemsets is useful because it not only reduce the number of patterns found but also can make the algorithm more than 10 times faster using the novel optimization called Length Upper-Bound Reduction.

High utility itemset mining has several applications such as discovering groups of items in transactions of a store that generate the most profit. A database containing utility information is a database where items can have quantities and a unit price. Although these algorithms are often presented in the context of market basket analysis, there exist other applications.

What is the input?

FHM+ takes as input a transaction database with utility information, a minimum utility threshold min_utility (a positive integer), a minimum pattern length (a positive number), and a maximum pattern length (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of FHM+ is the set of high utility itemsets having a utility no less than the min_utility threshold (a positive integer), and containing a number of items that is no less than the minimum pattern length and no greater the maximum pattern length, set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run FHM+ with a minimum utility of 30, a minimum length of 2 items, and a maximum length of 3 items, we obtain 6 high-utility itemsets respecting these constraints

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more, and that contain at least 2 items, and no more than 3 items..

Input file format

The input file format of FHM+ is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of FHM+ is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

1 3 5 #UTIL: 31
2 4 #UTIL: 30
2 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms.

The FHM algorithm was shown to be up to six times faster than HUI-Miner (also included in SPMF), especially for sparse datasets (see the performance section of the website for a comparison). The FHM+ algorithm is an optimized version of FHM for efficiently discovering high utility itemsets whe length constraints are used. It can be more than 10 times faster than FHM when length constraints are applied, thanks to a novel technique called Length Upper-bound Reduction.

Implementation details

The version offered in SPMF is the original implementation of FHM+.

Note that the input format is not exactly the same as described in the article. But it is equivalent.

Where can I get more information about the FHM+ algorithm?

This is the reference of the article describing the FHM+ algorithm:

Fournier-Viger, P., Lin, C.W., Duong, Q.-H., Dam, T.-L. (2016). FHM+: Faster High-Utility Itemset Mining using Length Upper-Bound Reduction . Proc. 29th Intern. Conf. on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA AIE 2016), Springer LNAI, to appear

Example 41 : Mining Correlated High-Utility Itemsets in a Database with Utility Information with the FCHM Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FCHM" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and the minbond parameter to 0.5 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FHM DB_utility.txt output.txt 30 0.5 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFCHM.java" in the package ca.pfv.SPMF.tests.

What is FCHM?

FCHM (Fournier-Viger et al., 2016)) is an algorithm for discovering correlated high-utility itemsets in a transaction databases containing utility information.

A limitation of traditional high utility itemset mining algorithms is that they may find many itemsets having a high utility but containing items that are weakly correlated (as shown in the FCHM paper). The FCHM addresses this issue by combining the idea of correlated pattern with high-utility pattern, to find high-utility itemsets where items are highly correlated. FCHM uses the bond measure to evaluate whether an itemset is a correlated itemset.

What is the input?

FCHM takes as input a transaction database with utility information, a minimum utility threshold min_utility (a positive integer), and a minbond threshold (a double number in the [0,1] interval). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of FCHM is the set of correlated high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user, and a bond no less than a minbond threshold also set by the user.

To explain what is a correlated high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility.

A correlated itemset is an itemset such that its bond is no less than a minbond threshold set by the user. The bond of an itemsets is the number of transactions containing the itemset divided by the number of transactions containing any of its items. The bond is a value in the [0,1] interval. A high value means a highly correlated itemset. Note that single items have by default a bond of 1. A correlated high-utility itemset is a high-utility itemset that is also a correlated itemset.

For example, if we run FHM with a minimum utility of 30 and minbond = 0.5, we obtain 3 correlated high-utility itemsets:

itemsets bond utility
{2 4} 0.5 30
{2 5} 0.75 31
{2 5 3} 0.6 37

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more, and containing items that are correlated (are likely to be bought together).

Input file format

The input file format of FCHM is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of FCHM is defined as follows. It is a text file, where each line represents a correlated high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. Then, there is a single space, followed by the keyword "#BOND: ", followed by the bond of the itemset. For example, we show below the output file for this example.

4 2 #UTIL: 30 #BOND: 0.5
2 5 #UTIL: 31 #BOND: 0.75
2 5 3 #UTIL: 37 #BOND: 0.6

For example, the first line indicates that the itemset {2, 4} has a utility of 30 and a bond of 0.5. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The FCHM algorithm is the first algorithm for mining correlated high-utility itemsets using the bond measure. It extends FHM, one of the fastest algorithm for high-utility itemsets mining.

Implementation details

Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the FCHM algorithm?

This is the reference of the article describing the FCHM algorithm:

Fournier-Viger, P., Lin, C. W., Dinh, T., Le, H. B. (2016). Mining Correlated High-Utility Itemsets Using the Bond Measure. Proc. 11 th International Conference on Hybrid Artificial Intelligence Systems (HAIS 2016), Springer LNAI, 14 pages, to appear.

Example 42 : Mining Frequent High-Utility Itemsets from a Database with Utility Information with the FHMFreq Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FHMFreq" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30, (5) set the minimum support to 0.4, and (6) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FHM DB_utility.txt output.txt 30 0.4 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFHMFreq.java" in the package ca.pfv.SPMF.tests.

What is FHMFreq?

FHM (Fournier-Viger et al., ISMIS 2014) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information. FHMFreq is a simple extension of FHM for discovering frequent high-utility itemsets (it combines frequent itemset mining with high-utility itemset mining).

High utility itemset mining has several applications such as discovering groups of items in transactions of a store that generate the most profit. A database containing utility information is a database where items can have quantities and a unit price. Although these algorithms are often presented in the context of market basket analysis, there exist other applications.

What is the input?

FHMFreq takes as input a transaction database with utility information, a minimum utility threshold min_utility (a positive integer), and a minimum support threshold minsup (a percentage value represented as a double in [0,1]). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of FHMFreq is the set of frequent high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user, and a support no less than the minsup threshold also set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. The support of an itemset is the number of transactions containing the itemset. For example, the support of itemset {1 4} is 2 transactions because it appears in transactions t1 and t3. The support of an itemset can also expressed as a percentage. For example, the support of itemset {1 4} is said to be 40% (or 0.4) because it appears in 2 out of five transactions in the database.

A frequent high utility itemset is an itemset such that its utility is no less than min_utility and that its support is no less than the minsup threshold. For example, if we run FHMFreq with a minimum utility of 30 and a minimum support of 40 %, we obtain 7 high-utility itemsets:

itemsets utility support
{2 4} 30 40 % (2 transactions)
{2 5} 31 60 % (3 transactions)
{1 3 5} 31 40 % (2 transactions)
{2 3 4} 34 40 % (2 transactions)
{2 3 5} 37 60 % (3 transactions)
{2 4 5} 36 40 % (2 transactions)
{2 3 4 5} 40 40 % (2 transactions)

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more, and appear in at least 2 transactions.

Input file format

The input file format of FHMFreq is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of FHMFreq is defined as follows. It is a text file, where each line represents a frequent high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. After, the keyword " #SUP: " appears and is followed by the support of the itemset. For example, we show below the output file for this example.

4 2 #UTIL: 30 #SUP: 2
4 2 5 #UTIL: 36 #SUP: 2
4 2 5 3 #UTIL: 40 #SUP: 2
4 2 3 #UTIL: 34 #SUP: 2
2 5 #UTIL: 31 #SUP: 3
2 5 3 #UTIL: 37 #SUP: 3
1 5 3 #UTIL: 31 #SUP: 2

For example, the first line indicates that the itemset {2, 4} has a utility of 30 and a support of two transactions. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms.

The FHMFreq algorithm here described is a simple extension of the FHM algorithm to add the minsup threshold as parmeter.

For high-utility itemset mining, the FHM algorithm was shown to be up to six times faster than HUI-Miner (also included in SPMF), especially for sparse datasets (see the performance section of the website for a comparison). But the EFIM algorithm (also included in SPMF) greatly outperforms FHM (see performance section of the website).

Implementation details

The version of FHMFreq offered in SPMF extends the original implementation of FHM.

Note that the input format is not exactly the same as described in the article. But it is equivalent.

Where can I get more information about the FHMFreq algorithm?

This is the reference of the article describing the FHM algorithm:

Fournier-Viger, P., Wu, C.-W., Zida, S., Tseng, V. (2014) FHM: A Faster High-Utility Itemset Mining Algorithm using Estimated Utility Co-occurrence Pruning. Proc. 21st International Symposium on Methodologies for Intelligent Systems (ISMIS 2014), Springer, LNAI, pp. 83-92

The FHMFreq algorithm is a simple extension of that algorithm.

Example 43 : Mining High-Utility Itemsets from a Database with Positive or Negative Unit Profit using the FHN Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FHN" algorithm, (2) select the input file "DB_NegativeUtility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FHN DB_NegativeUtility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_NegativeUtility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFHN_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is FHN?

FHN (Fournier-Viger et al, 2014) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information. It is an extension of the FHM algorithm designed for mining patterns in a transaction database where items may have negative unit profit values.

Items with negative values are interesting in real-life scenarios. Often in a retail store, items may be sold at a loss. If traditional high utility itemset mining algorithms such as Two-Phase, IHUP, UPGrowth, HUI-Miner and FHM are appied on such database, it was demonstrated that they may not discover the correct restults. To address this issue, algorithms such as HUINIV-Mine and FHN were proposed. At the time where FHN was proposed (2014), FHN is the state-of-the-art algorithm for mining high-tility itemsets with both positive and negative unit profit values.

This is the original implementation of FHN.

What is the input?

FHN takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 10 transactions (t1,t2...t10) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file "DB_NegativeUtility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 1 4 5 27 5 12 10
t2 2 3 4 36 -3 -4 36
t3 1 4 45 15 30
t4 1 5 15 5 10
t5 2 3 4 36 -3 -4 36
t6 2 3 5 20 -3 -2 20
t7 1 10 10
t8 1 4 21 15 6
t9 2 3 4 24 -3 -2 24
t10 1 5 15 5 10

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 1, 4 and 5. The amount of profit generated by the sale of each of these item is respectively 5 $, 12 $ and 10 $. The total amount of money spent in this transaction is 5 + 12 + 10 = 27 $.

What is the output?

The output of FHN is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 12 = 17 and the utility of {1 4} in transaction t3 is 15 + 30 = 45. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, plus the utility of {1 4} in t8, for a total of 17 + 45 + 21 = 83. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run FHN with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility ($)
{5} 50
{1 5} 45
{1} 55
{1 4} 83
{4} 144
{2 4} 87
{2 3 4} 77
{3 4} 86

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of FHN is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

1 4 5:27:5 12 10
2 3 4:36:-3 -4 36
1 4:45:15 30
1 5:15:5 10
2 3 4:36:-3 -4 36
2 3 5:20:-3 -2 20
1:10:10
1 4:21:15 6
2 3 4:24:-3 -2 24
1 5:15:5 10

Consider the first line. It means that the transaction {1, 4, 5} has a total utility of 27 and that items 1, 4and 5 respectively have a utility of 5, 12 and 10 in this transaction. The following lines follow the same format.

Output file format

The output file format of FHN is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

5 #UTIL: 50
5 1 #UTIL: 45
1 #UTIL: 55
1 4 #UTIL: 83
4 #UTIL: 144
4 2 #UTIL: 87
4 2 3 #UTIL: 77
4 3 #UTIL: 86

For example, the second line indicates that the itemset {1, 5} has a utility of 45. The other lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The FHN (2014) algorithm is up to 100 times faster than HUINIV-Mine, the previous state-of-the-art algorithm for high-utility itemset mining with negative unit profit.

Implementation details

The version of FHN in SPMF is the original implementation.

Where can I get more information about the FHN algorithm?

This is the reference of the article describing the FHN algorithm:

Fournier-Viger, P. (2014). FHN: Efficient Mining of High-Utility Itemsets with Negative Unit Profits. Proc. 10th International Conference on Advanced Data Mining and Applications (ADMA 2014), Springer LNCS 8933, pp. 16-29.

Example 44 : Mining High-Utility Itemsets from a Database with Positive or Negative Unit Profit using the HUINIV-Mine Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUINIV-Mine" algorithm, (2) select the input file "DB_NegativeUtility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUINIV-Mine DB_NegativeUtility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_NegativeUtility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHUINIVMine_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is HUINIV-Mine?

HUINIV-Mine is an algorithm for discovering high-utility itemsets in a transaction database containing utility information. It is an extension of the Two-Phase algorithm designed for mining patterns in a transaction database where items may have negative unit profit values.

Items with negative values are interesting in real-life scenarios. Often in a retail store, items may be sold at a loss. If traditional high utility itemset mining algorithms such as Two-Phase, IHUP, UPGrowth, HUI-Miner and FHM are appied on such database, it was demonstrated that they may not discover the correct restults. To address this issue, the HUINIV-Mine algorithm was proposed. However, faster algorithms now exists, such as FHN, also offered in SPMF.

What is the input?

HUINIV-Mine takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 10 transactions (t1,t2...t10) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file "DB_NegativeUtility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 1 4 5 27 5 12 10
t2 2 3 4 36 -3 -4 36
t3 1 4 45 15 30
t4 1 5 15 5 10
t5 2 3 4 36 -3 -4 36
t6 2 3 5 20 -3 -2 20
t7 1 10 10
t8 1 4 21 15 6
t9 2 3 4 24 -3 -2 24
t10 1 5 15 5 10

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 1, 4 and 5. The amount of profit generated by the sale of each of these item is respectively 5 $, 12 $ and 10 $. The total amount of money spent in this transaction is 5 + 12 + 10 = 27 $.

What is the output?

The output of HUINIV-Mineis the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 12 = 17 and the utility of {1 4} in transaction t3 is 15 + 30 = 45. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, plus the utility of {1 4} in t8, for a total of 17 + 45 + 21 = 83. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run HUINIV-Mine with a minimum utility of 30, we obtain 8 high-utility itemsets:

itemsets utility ($)
{5} 50
{1 5} 45
{1} 55
{1 4} 83
{4} 144
{2 4} 87
{2 3 4} 77
{3 4} 86

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Input file format

The input file format of HUINIV-Mine is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

1 4 5:27:5 12 10
2 3 4:36:-3 -4 36
1 4:45:15 30
1 5:15:5 10
2 3 4:36:-3 -4 36
2 3 5:20:-3 -2 20
1:10:10
1 4:21:15 6
2 3 4:24:-3 -2 24
1 5:15:5 10

Consider the first line. It means that the transaction {1, 4, 5} has a total utility of 27 and that items 1, 4and 5 respectively have a utility of 5, 12 and 10 in this transaction. The following lines follow the same format.

Output file format

The output file format of HUINIV-Mine is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

5 #UTIL: 50
5 1 #UTIL: 45
1 #UTIL: 55
1 4 #UTIL: 83
4 #UTIL: 144
4 2 #UTIL: 87
4 2 3 #UTIL: 77
4 3 #UTIL: 86

For example, the second line indicates that the itemset {1, 5} has a utility of 45. The other lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The HUINIV-Mine is the first algorithm for high-utility itemset mining with negative unit profit. However, faster algorithms have now been proposed such as FHN (2014), also offered in SPMF.

Where can I get more information about the HUINIV-Mine algorithm?

This is the reference of the article describing the HUINIV-Mine algorithm:

Chu, Chun-Jung, Vincent S. Tseng, and Tyne Liang. "An efficient algorithm for mining high utility itemsets with negative item values in large databases." Applied Mathematics and Computation 215.2 (2009): 767-778.

Example 45 : Mining On-Shelf High-Utility Itemsets from a Transaction Database using the FOSHU Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FOSHU" algorithm, (2) select the input file "DB_FOSHU.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility ratio to 0.8 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FOSHU DB_FOSHU.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_FOSHU.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestFOSHU_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is FOSHU?

FOSHU (Fournier-Viger et al, 2015) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information and information about the time periods where items are sold. The task of on-shelf high-utility itemset mining is an extension of the task of high utility itemset mining.

The FOSHU algorithm for on-shelf-high-utility itemset mining is interesting because it addresses two limitations of high-utility itemset mining algorithms. First, most algorithms cannot handle databases where items may have negative unit profit/weight. But such items often occur in real-life transaction databases. For example, it is common that a retail store will sell items at a loss to stimulate the sale of other related items or simply to attract customers to their retail location. If classical HUIM algorithms are applied on database containing items with negative unit profit, they can generate an incomplete set of high-utility itemsets. Second, most algorithms consider that items have the same shelf time, i.e. that all item are on sale for the same time period. However, in real-life some items are only sold during some short time period (e.g. the summer). Algorithms ignoring the shelf time of items have a bias toward items having more shelf time since they have more chance to generate a high profit.

FOSHU is the state-of-the-art algorithm for on-shelf high-utility itemset mining. It was shown to outperform TS-HOUN by up to three orders of magnitude in terms of execution time.

This is the original implementation of FOSHU.

What is the input?

FOSHU takes as input a transaction database with information about the utility of items and their shelf time time, and a minimum utility threshold min_utility ratio (a positive double value in the [0,1] interval). For example, let's consider the following database consisting of 5 transactions (t1,t2, ..., t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_FOSHU.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Transaction Items Transaction utility (positive) Item utilities for this transaction Time period
t1 1 3 4 3 -5 1 2 0
t2 1 3 5 7 17 -10 6 6 5 0
t3 1 2 3 4 5 6 25 -5 4 1 12 3 5 1
t4 2 3 4 5 20 8 3 6 3 1
t5 2 3 5 7 11 4 2 3 2 2

Each line of the database represents a transaction and contains the following information:

  • a set of items (the second column of the table),
  • the sum of the utilities (e.g. profit) of items having positive utilities in this transaction (the third column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the fourth column of the table).
  • the time period where this transaction occurred (the fifth column).

Note that the value in the third column for each line is the sum of the positive values in the fourth column. Moreoever, note that utility values may be positive or negative integers. Time periods are values numbered 0,1,2,3..., which may represent for example periods such as "summer", "fall", "winter" and "spring".

What are real-life examples of such a database? There are several applications in real life. The main application is for customer transaction databases. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 1, 3 and 4. The amount of profit generated by the sale of each of these item is respectively -5 $, 1 $ and 2 $. The total amount of money spent in this transaction is -5 + 1 + 2 = 3 $. This transaction was done during time period "0", which may for example represents the summer.

What is the output?

The output of the FOSHU algorithm is the set of on-shelf high utility itemsets having a relative utility no less than the min_utility_ratio threshold set by the user. To explain what is an on-shelf high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1, 3, 4} in transaction t1 is -5 + 1 + 2 = 3, and the utility of {1, 3, 4} in transaction t3 is -5 + 1 + 12 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1, 3, 4} in the database is the utility of {1, 3, 4} in t1 plus the utility of {1, 3, 4} in t3, for a total of -2 + 8 = 6. The relative utility of an itemset is the utility of that itemset divided by the sum of the transaction utilities for the time period where the itemset was sold (including the negative utilities. For example, itemset {1, 3, 4} was sold in time periods "0" and "1". The total utility of time period "0" and "1" is 5 + 40 = 45. Thus, the relative utility of {1, 3, 4} is 6 / 45 = 0.13. The relative utility can be interpreted as a ratio of the profit generated by a given itemset during the time period when it was sold.

A on-shelf high utility itemset is an itemset such that its relative utility is no less than min_utility_ratio. For example, if we run FOSHU with a minimum utility of 0.8, we obtain the following on-shelf high-utility itemsets:

itemsets utility ($) relative utility
{2, 5, 7} 9 $ 0.81
{2, 3, 5, 7} 11 $ 1
{5, 7} 16 $ 1
{3, 5, 7} 24 $ 1.5
{1, 3, 5, 7} 7 $ 1.4
{3, 7} 15 $ 0.9375
{2, 4, 5} 36 $ 0.9
{2, 3, 4, 5} 40 $ 1
{2, 3, 4} 34 $ 0.85

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a ratio of at least 0.8 on the total profit during the time period when they were sold.

Input file format

The input file format of FOSHU is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.
  • Fourth, the symbol ":" appears and is followed by a positive integer such as 0,1,2.... indicating the time period of the transaction

For example, for the previous example, the input file is defined as follows:

1 3 4:3:-5 1 2:0
1 3 5 7:17:-10 6 6 5:0
1 2 3 4 5 6:25:-5 4 1 12 3 5:1
2 3 4 5:20:8 3 6 3:1
2 3 5 7:11:4 2 3 2:2

Consider the first line. It means that the transaction {1,3, 4} has a total utility of 3 and that items 1, 3 and 4 respectively have a utility of -5, 1 and 2 in this transaction. The following lines follow the same format.

Output file format

The output file format of FOSHUis defined as follows. It is a text file, where each line represents a on-shelf high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. Then, the keyword "#RUTIL:" appears followed by the relative utility of this itemset. For example, we show below the output file for this example.

7 2 5 #UTIL: 9 #RUTIL: 0.8181818181818182
7 2 5 3 #UTIL: 11 #RUTIL: 1.0
7 5 #UTIL: 16 #RUTIL: 1.0
7 5 3 #UTIL: 24 #RUTIL: 1.5
7 5 3 1 #UTIL: 7 #RUTIL: 1.4
7 3 #UTIL: 15 #RUTIL: 0.9375
4 2 5 #UTIL: 36 #RUTIL: 0.9
4 2 5 3 #UTIL: 40 #RUTIL: 1.0
4 2 3 #UTIL: 34 #RUTIL: 0.85

For example, the second line indicates that the itemset {2, 3, 5, 7} has a utility of 11 $ and a relative utility of 1. The other lines follows the same format.

Performance

On-shelf high utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, on-shelf high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The FOSHU (2015) algorithm is up to 1000 times faster than TS-HOUN, the previous state-of-the-art algorithm for on-shelf high-utility itemset mining.

Implementation details

The version of FOSHU offered in SPMF is the original implementation.

Where can I get more information about the FOSHU algorithm?

This is the reference of the article describing the FOSHU algorithm:

Fournier-Viger, P., Zida, S. (2015). FOSHU: Faster On-Shelf High Utility Itemset Mining– with or without negative unit profit. Proc. 30th Symposium on Applied Computing (ACM SAC 2015). ACM Press, pp. 857-864.

Example 46 : Mining On-Shelf High-Utility Itemsets from a Transaction Database using the TS-HOUN Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "TS-HOUN" algorithm, (2) select the input file "DB_FOSHU.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility ratio to 0.8 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run TS-HOUN DB_FOSHU.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_FOSHU.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestTSHOUN_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is TS-HOUN?

TS-HOUN (Lan et al, 2014) is an algorithm for discovering high-utility itemsets in a transaction database containing utility information and information about the time periods where items are sold. The task of on-shelf high-utility itemset mining is an extension of the task of high utility itemset mining.

The TS-HOUN algorithm for on-shelf-high-utility itemset mining is interesting because it addresses two limitations of high-utility itemset mining algorithms. First, most algorithms cannot handle databases where items may have negative unit profit/weight. But such items often occur in real-life transaction databases. For example, it is common that a retail store will sell items at a loss to stimulate the sale of other related items or simply to attract customers to their retail location. If classical HUIM algorithms are applied on database containing items with negative unit profit, they can generate an incomplete set of high-utility itemsets. Second, most algorithms consider that items have the same shelf time, i.e. that all item are on sale for the same time period. However, in real-life some items are only sold during some short time period (e.g. the summer). Algorithms ignoring the shelf time of items have a bias toward items having more shelf time since they have more chance to generate a high profit.

TS-HOUN is the first algorithm for on-shelf high utility itemset mining with both positive and negative profit values. However, it was outperformed by FOSHU (also offered in SPMF). FOSHU was shown to outperform TS-HOUN by up to three orders of magnitude in terms of execution time (see "Performance" section of this website for more details).

This is the original implementation of FOSHU.

What is the input?

TS-HOUN takes as input a transaction database with information about the utility of items and their shelf time time, and a minimum utility threshold min_utility ratio (a positive double value in the [0,1] interval). For example, let's consider the following database consisting of 5 transactions (t1,t2, ..., t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_FOSHU.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Transaction Items Transaction utility (positive) Item utilities for this transaction Time period
t1 1 3 4 3 -5 1 2 0
t2 1 3 5 7 17 -10 6 6 5 0
t3 1 2 3 4 5 6 25 -5 4 1 12 3 5 1
t4 2 3 4 5 20 8 3 6 3 1
t5 2 3 5 7 11 4 2 3 2 2

Each line of the database represents a transaction and contains the following information:

  • a set of items (the second column of the table),
  • the sum of the utilities (e.g. profit) of items having positive utilities in this transaction (the third column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the fourth column of the table).
  • the time period where this transaction occurred (the fifth column).

Note that the value in the third column for each line is the sum of the positive values in the fourth column. Moreoever, note that utility values may be positive or negative integers. Time periods are values numbered 0,1,2,3..., which may represent for example periods such as "summer", "fall", "winter" and "spring".

What are real-life examples of such a database? There are several applications in real life. The main application is for customer transaction databases. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 1, 3 and 4. The amount of profit generated by the sale of each of these item is respectively -5 $, 1 $ and 2 $. The total amount of money spent in this transaction is -5 + 1 + 2 = 3 $. This transaction was done during time period "0", which may for example represents the summer.

What is the output?

The output of TS-HOUN algorithm is the set of on-shelf high utility itemsets having a relative utility no less than the min_utility_ratio threshold set by the user. To explain what is an on-shelf high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1, 3, 4} in transaction t1 is -5 + 1 + 2 = 3, and the utility of {1, 3, 4} in transaction t3 is -5 + 1 + 12 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1, 3, 4} in the database is the utility of {1, 3, 4} in t1 plus the utility of {1, 3, 4} in t3, for a total of -2 + 8 = 6. The relative utility of an itemset is the utility of that itemset divided by the sum of the transaction utilities for the time period where the itemset was sold (including the negative utilities. For example, itemset {1, 3, 4} was sold in time periods "0" and "1". The total utility of time period "0" and "1" is 5 + 40 = 45. Thus, the relative utility of {1, 3, 4} is 6 / 45 = 0.13. The relative utility can be interpreted as a ratio of the profit generated by a given itemset during the time period when it was sold.

A on-shelf high utility itemset is an itemset such that its relative utility is no less than min_utility_ratio. For example, if we run TS-HOUN with a minimum utility of 0.8, we obtain the following on-shelf high-utility itemsets:

itemsets utility ($) relative utility
{2, 5, 7} 9 $ 0.81
{2, 3, 5, 7} 11 $ 1
{5, 7} 16 $ 1
{3, 5, 7} 24 $ 1.5
{1, 3, 5, 7} 7 $ 1.4
{3, 7} 15 $ 0.9375
{2, 4, 5} 36 $ 0.9
{2, 3, 4, 5} 40 $ 1
{2, 3, 4} 34 $ 0.85

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a ratio of at least 0.8 on the total profit during the time period when they were sold.

Input file format

The input file format of TS-HOUN is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.
  • Fourth, the symbol ":" appears and is followed by a positive integer such as 0,1,2.... indicating the time period of the transaction

For example, for the previous example, the input file is defined as follows:

1 3 4:3:-5 1 2:0
1 3 5 7:17:-10 6 6 5:0
1 2 3 4 5 6:25:-5 4 1 12 3 5:1
2 3 4 5:20:8 3 6 3:1
2 3 5 7:11:4 2 3 2:2

Consider the first line. It means that the transaction {1,3, 4} has a total utility of 3 and that items 1, 3 and 4 respectively have a utility of -5, 1 and 2 in this transaction. The following lines follow the same format.

Output file format

The output file format of TS-HOUN is defined as follows. It is a text file, where each line represents a on-shelf high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. Then, the keyword "#RUTIL:" appears followed by the relative utility of this itemset. For example, we show below the output file for this example.

7 2 5 #UTIL: 9 #RUTIL: 0.8181818181818182
7 2 5 3 #UTIL: 11 #RUTIL: 1.0
7 5 #UTIL: 16 #RUTIL: 1.0
7 5 3 #UTIL: 24 #RUTIL: 1.5
7 5 3 1 #UTIL: 7 #RUTIL: 1.4
7 3 #UTIL: 15 #RUTIL: 0.9375
4 2 5 #UTIL: 36 #RUTIL: 0.9
4 2 5 3 #UTIL: 40 #RUTIL: 1.0
4 2 3 #UTIL: 34 #RUTIL: 0.85

For example, the second line indicates that the itemset {2, 3, 5, 7} has a utility of 11 $ and a relative utility of 1. The other lines follows the same format.

Performance

On-shelf high utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, on-shelf high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms.TS-HOUN (2014) is the first algorithm for on-shelf high utility itemset mining with both positive and negative profit values. However, it was outperformed by FOSHU (2015) (also offered in SPMF). FOSHU was shown to outperform TS-HOUN by up to three orders of magnitude in terms of execution time (see "Performance" section of this website for more details).

Where can I get more information about the TS-HOUN algorithm?

This is the reference of the article describing the TS-HOUN algorithm:

G.-C. Lan, T.-P. Hong, J.-P. Huang and V.S. Tseng. On-shelf utility mining with negative item values. In Expert Systems with Applications. 41:3450–3459, 2014.

Example 47 : Incremental High-Utility Itemset Mining in a Database with Utility Information with the EIHI Algorithm

How to run this example?

  • This algorithm is not offered in the graphical user interface of SPMF.
  • If you are using the source code version of SPMF, launch the file "MainTestEIHI.java" in the package ca.pfv.SPMF.tests.

What is EIHI?

EIHI (Fournier-Viger et al., 2015) is an algorithm for maintaining high-utility itemsets in a transaction database containing utility information that is updated incrementally by inserting new transactions. This task called "incremental high-utility itemset mining" is a generalization of the task of high utility itemset mining, where the database is not assumed to be static.

What is the input?

EIHI takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 4 transactions (t1,t2...t4) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_incremental1.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

The EIHI algorithm is an incremental algorithm, which means that it can efficiently update the result when new transactions are inserted into the database. In this example, we will consider that a new transaction is inserted into the database, as follows:

t5 3 5 2 7 11 2 3 4 2

This transaction is provided in the file "DB_incremental2.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

What is the output?

The output of EIHI is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, consider the initial database containing transactions t1, t2, t3 and t4. In this database, the utility of {1 4} is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run EIHI with a minimum utility of 30 on the initial database containing t1,t2,t3 and t4, we obtain 6 high-utility itemsets:

itemsets utility
{2 4} 30
{1 3 5} 31
{2 3 4} 34
{2 4 5} 36
{2 3 4 5} 40
{1 2 3 4 5 6} 30

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Now, EIHI is an incremental. It is designed to update the set of high-utility itemsets when new transactions are inserted. For example, consider that transaction t5 is now inserted. The results is thus updated as follows, where 8 high-utility itemsets are found:

itemsets utility
{2 4} 30
{2 5} 31
{1 3 5} 31
{2 3 4} 34
{2 3 5} 37
{2 4 5} 36
{2 3 4 5} 40
{1 2 3 4 5 6} 30

Input file format

The input file format of EIHI is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file "DB_incremental1.txt" is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5

And the input file "DB_incremental2.txt" is defined as follows:

3 5 2 7:11:2 3 4 2

Consider the first line of the file "DB_incremental1.txt". It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of EIHI is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file after all transactions have been processed from both files.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms.

The EIHI algorithm was shown to be up to 100 times faster than HUI-LIST-INS (also included in SPMF), the previous state-of-the-art algorithm for maintaining high-utility itemsets in transactions databases where transaction insertions are performed.

Implementation details

The version offered in SPMF is the original implementation of EIHI.

Note that the input format is not exactly the same as described in the article. But it is equivalent.

Note also, that a file "MainTestEIHI_Xruns.java" is provided in the package "ca.pfv.spmf.tests". This file can be used to run experiments such as those provided in the article proposing EIHI where a different number of updates is varied on some datasets. This example uses a single file as input and divide it into several parts. Then, the algorithm is incrementally run by processing each part of the file one after the other.

Where can I get more information about the EIHI algorithm?

This is the reference of the article describing the EIHI algorithm:

Fournier-Viger, P., Lin, J. C.-W., Gueniche, T., Barhate, P. (2015). Efficient Incremental High Utility Itemset Mining. Proc. 5th ASE International Conference on Big Data (BigData 2015), to appear.

Example 48 : Incremental High-Utility Itemset Mining in a Database with Utility Information with the HUI-LIST-INS Algorithm

How to run this example?

  • This algorithm is not offered in the graphical user interface of SPMF.
  • If you are using the source code version of SPMF, launch the file "MainTestHUI_LIST_INS.java" in the package ca.pfv.SPMF.tests.

What is HUI-LIST-INS?

HUI-LIST-INS (Lin et al., 2014) is an algorithm for maintaining high-utility itemsets in a transaction database containing utility information that is updated incrementally by inserting new transactions. This task called "incremental high-utility itemset mining" is a generalization of the task of high utility itemset mining, where the database is not assumed to be static.

Note that the faster algorithm EIHI is also offered in SPMF.

What is the input?

HUI-LIST-INS takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 4 transactions (t1,t2...t4) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_incremental1.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

The HUI-LIST-INS algorithm is an incremental algorithm, which means that it can efficiently update the result when new transactions are inserted into the database. In this example, we will consider that a new transaction is inserted into the database, as follows:

t5 3 5 2 7 11 2 3 4 2

This transaction is provided in the file "DB_incremental2.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

What is the output?

The output of HUI-LIST-INS is the set of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility itemset, it is necessary to review some definitions. An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, consider the initial database containing transactions t1, t2, t3 and t4. In this database, the utility of {1 4} is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility For example, if we run HUI-LIST-INS with a minimum utility of 30 on the initial database containing t1,t2,t3 and t4, we obtain 6 high-utility itemsets:

itemsets utility
{2 4} 30
{1 3 5} 31
{2 3 4} 34
{2 4 5} 36
{2 3 4 5} 40
{1 2 3 4 5 6} 30

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more.

Now, HUI-LIST-INS is an incremental. It is designed to update the set of high-utility itemsets when new transactions are inserted. For example, consider that transaction t5 is now inserted. The results is thus updated as follows, where 8 high-utility itemsets are found:

itemsets utility
{2 4} 30
{2 5} 31
{1 3 5} 31
{2 3 4} 34
{2 3 5} 37
{2 4 5} 36
{2 3 4 5} 40
{1 2 3 4 5 6} 30

Input file format

The input file format of HUI-LIST-INS is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file "DB_incremental1.txt" is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5

And the input file "DB_incremental2.txt" is defined as follows:

3 5 2 7:11:2 3 4 2

Consider the first line of the file "DB_incremental1.txt". It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of HUI-LIST-INS is defined as follows. It is a text file, where each line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file after all transactions have been processed from both files.

2 4 #UTIL: 30
2 5 #UTIL: 31
1 3 5 #UTIL: 31
2 3 4 #UTIL: 34
2 3 5 #UTIL: 37
2 4 5 #UTIL: 36
2 3 4 5 #UTIL: 40
1 2 3 4 5 6 #UTIL: 30

For example, the first line indicates that the itemset {2, 4} has a utility of 30. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms.

The EFIM algorithm was shown to be up to 100 times faster than HUI-LIST-INS (also included in SPMF) for maintaining high-utility itemsets in transactions databases where transaction insertions are performed.

Implementation details

Note that the input format is not exactly the same as described in the article. But it is equivalent.

Note also, that a file "MainTestHUI_LIST_INS_Xruns.java" is provided in the package "ca.pfv.spmf.tests". This file can be used to run experiments such as those provided in the article proposing HUI-LIST-INS where a different number of updates is varied on some datasets. This example uses a single file as input and divide it into several parts. Then, the algorithm is incrementally run by processing each part of the file one after the other.

Where can I get more information about the HUI-LIST-INS algorithm?

This is the reference of the article describing the HUI-LIST-INS algorithm:

J. C.-W. Lin, W. Gan, T.P. Hong, J. S. Pan, Incrementally Updating High-Utility Itemsets with Transaction Insertion. In: Proc. 10th Intern. Conference on Advanced Data Mining and Applications (ADMA 2014), Springer (2014)

Example 49 : Mining Closed High-Utility Itemsets from a transaction database with utility information using the CHUI-Miner Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "EFIM-Closed" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run EFIM-Closed DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestEFIM_Closed_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is EFIM-Closed?

EFIM-Closed (Fournier-Viger et al., 2016) is an algorithm for discovering closed high-utility itemsets in a transaction database containing utility information.

There has been many work on the design of algorithms for high-utility itemset mining. However, a limitation of many high-utility itemset mining algorithms is that they output too many itemsets. As a result, it may be inconvenient for a user to analyze the result of traditional high utility itemset mining algorithms. As a solution, algorithms have been designed to discover only the high-utility itemsets that are closed. The concept of closed itemset was previously introduced in frequent itemset mining. An itemset is closed if it has no subset having the same support (frequency) in the database. In terms of application to transaction databases, the concept of closed itemset can be understood as any itemset that is the largest set of items bought by a given set of customers. For more details, you may look at the paper about EFIM-Closed. It provides more details about the motivation for mining closed high-utility itemsets. Other popular alternative algorithms for closed high-utility itemsets mining are CHUI-Miner (2015, also offered in SPMF), and CHUD (2011,2013, currently not offered in SPMF).

What is the input?

EFIM-Closed takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of EFIM-Closed is the set of closed high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a closed high utility itemset, it is necessary to review some definitions.

An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility.

To explain what is a closed itemset it is necessary to review a few definitions.

The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3, 5} has a support of 2 because it appears in three transactions from the database (t1 and t4). A closed is an itemset X such that there does not exist an itemset Y strictly included in X that has the same support. For example, itemset {1, 3, 5} is a closed itemset.

A closed high utility itemset (CHUI) is a high-utility itemset that is a closed itemset.

For example, if we run EFIM-Closed with a minimum utility of 30 we obtain 4 closed high-utility itemsets:

itemsets utility support
{1, 2, 3, 4, 5, 6} 30 1 transaction
{2, 3, 4, 5} 40 2 transactions
{2, 3, 5} 37 3 transactions
{1, 3, 5} 31 2 transactions

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more, and that are maximal sets of items in common for a group of customers.

Input file format

The input file format of EFIM-Closed is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of EFIM-Closed is defined as follows. It is a text file, where each line represents a closed high utility itemsets. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUPPORT:" appears and is followed by the support of the itemset. Then, the keyword #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

6 4 2 1 5 3 #SUP: 1 #UTIL: 30
4 3 2 5 #SUP: 2 #UTIL: 40
2 5 3 #SUP: 3 #UTIL: 37
1 3 5 #SUP: 2 #UTIL: 31

For example, the third line indicates that the itemset {2, 3, 5} has a support of 3 transactions and a utility of 37$. The other lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The EFIM-Closed algorithm was proposed in 2016 to discover only the high-utility itemsets that are closed itemsets. It is generally faster to mine closed high-utility itemsets than discovering all high-utility itemsets. Thus, this algorithm can in some cases outperform algorithms such as FHM and HUI-Miner, who discover all high-utility itemsets. The EFIM-Closed algorithm was shown to outperform the original algorithm for mining closed high-utility itemsets named CHUD algorithm (published in the proceedings of the ICDM 2011 conference).

Implementation details

This is an implementation of EFIM-Closed, implemented by P. Fournier-Viger. This is an alternative implementation that was not used in the paper. The main differences with the implementation in the paper is that this implementation (1) does not calculate utility-unit arrays (see the paper) and (2) adds the EUCP optimizations introduced in the FHM algorithm.

In the source code version of SPMF, there are two examples of using EFIM-Closed in the package ca.pfv.spmf.tests. The first one is MainTestEFIM_Closed_saveToFile, which saves the result to an output file. The second one is MainTestEFIM_Closed_saveToMemory, which saves the result to memory.

Where can I get more information about the EFIM_Closed algorithm?

This is the reference of the article describing the EFIM_Closed algorithm:

Fournier-Viger, P., Zida, S. Lin, C.W., Wu, C.-W., Tseng, V. S. (2016). EFIM-Closed: Fast and Memory Efficient Discovery of Closed High-Utility Itemsets. Proc. 12th Intern. Conference on Machine Learning and Data Mining (MLDM 2016). Springer, LNAI, 15 pages, to appear

Example 50 : Mining Closed High-Utility Itemsets from a transaction database with utility information using the CHUI-Miner Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "CHUI-Miner" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run CHUI-Miner DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestCHUIMiner_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is CHUI-Miner?

CHUI-Miner (Wu et al., 2014) is an algorithm for discovering closed high-utility itemsets in a transaction database containing utility information.

There has been many work on the topic of high-utility itemset mining. A limitation of many high-utility itemset mining algorithms is that they generate too much itemsets as output. The CHUI-Miner algorithm was designed to discover only the high-utility itemsets that are closed. The concept of closed itemset was previously introduced in frequent itemset mining. An itemset is closed if it has no subset having the same support (frequency) in the database. In terms of application to transaction database, the concept of closed itemset can be understood as any itemset that is the largest set of items bought in common by a given set of customers. For more details, see the paper by Wu et al. (2015). It provides a more details about the motivation for mining closed high-utility itemsets.

What is the input?

CHUI-Miner takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of CHUI-Miner is the set of closed high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a closed high utility itemset, it is necessary to review some definitions.

An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility.

To explain what is a closed itemset it is necessary to review a few definitions.

The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 3, 5} has a support of 2 because it appears in three transactions from the database (t1 and t4). A closed is an itemset X such that there does not exist an itemset Y strictly included in X that has the same support. For example, itemset {1, 3, 5} is a closed itemset.

A closed high utility itemset (CHUI) is a high-utility itemset that is a closed itemset.

For example, if we run CHUI-Miner with a minimum utility of 30 we obtain 4 closed high-utility itemsets:

itemsets utility support
{1, 2, 3, 4, 5, 6} 30 1 transaction
{2, 3, 4, 5} 40 2 transactions
{2, 3, 5} 37 3 transactions
{1, 3, 5} 31 2 transactions

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 30 $ or more, and that are maximal sets of items in common for a group of customers.

Input file format

The input file format of CHUI-Miner is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of CHUI-Miner is defined as follows. It is a text file, where each line represents a closed high utility itemsets. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUPPORT:" appears and is followed by the support of the itemset. Then, the keyword #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

6 4 2 1 5 3 #SUP: 1 #UTIL: 30
4 3 2 5 #SUP: 2 #UTIL: 40
2 5 3 #SUP: 3 #UTIL: 37
1 3 5 #SUP: 2 #UTIL: 31

For example, the third line indicates that the itemset {2, 3, 5} has a support of 3 transactions and a utility of 37$. The other lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The CHUI-Miner algorithm was proposed in 2015 to discover only the high-utility itemsets that are closed itemset. It is generally faster than discovering all high-utility itemsets. Thus, this algorithm can in some cases outperform algorithms such as FHM and HUI-Miner, who discover all high-utility itemsets. The CHUI-Miner algorithm is an improved version of the CHUD algorithm published in the proceedings of the ICDM 2011 conference.

Implementation details

This is an implementation of CHUI-Miner, implemented by P. Fournier-Viger. This is an alternative implementation that was not used in the paper. The main differences with the implementation in the paper is that this implementation (1) does not calculate utility-unit arrays (see the paper) and (2) adds the EUCP optimizations introduced in the FHM algorithm.

In the source code version of SPMF, there are two examples of using CHUI-Miner in the package ca.pfv.spmf.tests. The first one is MainTestCHUIMiner_saveToFile, which saves the result to an output file. The second one is MainTestCHUIMiner_saveToMemory, which saves the result to memory.

Where can I get more information about the CHUI-Miner algorithm?

This is the reference of the article describing the CHUI-Miner algorithm:

Wu, C.W., Fournier-Viger, P., Gu, J.-Y., Tseng, V.S. (2015). Mining Closed+ High Utility Itemsets without Candidate Generation. Proc. 2015 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2015), pp. 187-194.

Example 51: Mining Generators of High-Utility Itemsets from a transaction database with utility information using the GHUI-Miner Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "GHUI-Miner" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 20 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run GHUI-Miner DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestGHUIMiner_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is GHUI-Miner?

GHUI-Miner (Fournier-Viger et al., 2014) is an algorithm for discovering generators of high-utility itemsets in a transaction database containing utility information.

There has been quite a huge amount of work on the topic of high-utility itemset mining. A limitation of several high-utility itemset mining algorithms is that they generate too much results. The GHUI-Miner algorithm was designed to discover only the generators of high-utility itemsets. The concept of generator was previously introduced in frequent itemset mining. An itemset is a generator if it has no subset having the same support (frequency) in the database. An itemset is closed if it has no superset having the same support (frequency) in the database. In terms of application to transaction database, the concept of generator can be understood as any itemset that is the smallest set of items bought in common by a given set of customers, while a closed itemset is the maximal set of items. Generators have shown to be more useful than closed or maximal itemsets in the field of pattern mining for various tasks such as classification. The GHUI-Miner algorithm discovers all generators of high-utility itemsets, that is generators that (1) are high-utility itemsets or (2) have a superset that is a high-utility itemset and has the same support.

For more details, see the paper by Fournier-Viger (2014). It provides a lot of details about the motivation for mining generators of high-utility itemsets.

This is the original implementation of GHUI-Miner.

What is the input?

GHUI-Miner takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of GHUI-Miner is the set of generators of high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility generator itemsets, it is necessary to review some definitions.

An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility.

To explain what is a generator, it is necessary to review a few definitions.

The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 5} has a support of 2 because it appears in three transactions from the database (t1 and t4). A generator is an itemset X such that there does not exist an itemset Y strictly included in X that has the same support. For example, itemset {1, 5} is a generator.

A generator of high-utility itemsets (HUG) is a generator itemset that (1) is a high-utility itemsets or (2) has a superset that is a high-utility itemset and has the same support.

For example, if we run GHUI-Miner with a minimum utility of 30, we obtain 7 generator of high-utility itemsets:

itemsets utility support
{2} 22 3 transactions
{2, 4} 30 2 transactions
{1 5} 24 2 transactions
{6}
5
1 transaction
{4, 5}
18
2 transactions
{1, 4, 5}
20
1 transaction
{1, 2}
15
1 transaction

If the database is a transaction database from a retail store, we could interpret each itemset found as the smallest set of items common to a group of customers that has bought a given high-utility itemset.

Input file format

The input file format of GHUI-Miner is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of GHUI-Miner is defined as follows. It is a text file, where each line represents a generator of high-utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUPPORT:" appears and is followed by the support of the itemset. Then, the keyword #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

6 #SUP: 1 #UTIL: 5
2 #SUP: 3 #UTIL: 22
4 2 #SUP: 2 #UTIL: 30
4 5 #SUP: 2 #UTIL: 18
4 1 5 #SUP: 1 #UTIL: 20
2 1 #SUP: 1 #UTIL: 15
1 5 #SUP: 2 #UTIL: 24

For example, the third line indicates that the itemset {2, 4} has a support of 2 transactions and a utility of 30$. The other lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The GHUI-Miner algorithm was proposed in 2014 to discover only the high-utility itemsets that are generators. It is generally faster than discovering all high-utility itemsets. Thus, this algorithm can outperform algorithms such as FHM and HUI-Miner, who discover all high-utility itemsets. This implementation of GHUI-Miner relies on the CHUI-Miner algorithm for discovering closed high-utility itemsets (a necessary step to find GHUIs efficiently)

Implementation details

This is the original implementation of GHUI-Miner.

Where can I get more information about the GHUI-Miner algorithm?

This is the reference of the article describing the GHUI-Miner algorithm:

Fournier-Viger, P., Wu, C.W., Tseng, V.S. (2014). Novel Concise Representations of High Utility Itemsets using Generator Patterns. Proc. 10th Intern. Conference on Advanced Data Mining and Applications (ADMA 2014), Springer LNCS 8933, pp. 30-43.

Note that in this article, another algorithm called HUG-Miner is also proposed. It is a different algorithm, which is also offered in SPMF.

Example 52 : Mining High-Utility Generator Itemsets from a transaction database with utility information using the HUG-Miner Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUG-Miner" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 20 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUG-Miner DB_utility.txt output.txt 20 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTest_HUGMINER_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is HUG-Miner?

HUG-Miner (Fournier-Viger et al., 2014) is an algorithm for discovering high-utility generator itemsets in a transaction database containing utility information.

There has been quite a huge amount of work on the topic of high-utility itemset mining. A limitation of several high-utility itemset mining algorithms is that they generate too much results. The HUG-Miner algorithm was designed to discover only the high-utility itemsets that are generators. The concept of generator was previously introduced in frequent itemset mining. An itemset is a generator if it has no subset having the same support (frequency) in the database. In terms of application to transaction database, the concept of generator can be understood as any itemset that is the smallest set of items bought in common by a given set of customers. Generators have shown to be more useful than closed or maximal itemsets in the field of pattern mining for various tasks such as classification. For more details, see the paper by Fournier-Viger (2014). It provides a lot of details about the motivation for mining High-utility genrator itemsets.

This is the original implementation of HUG-Miner.

What is the input?

HUG-Miner takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of HUG-Miner is the set of high utility generator itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a high utility generator itemsets, it is necessary to review some definitions.

An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility.

To explain what is a generator, it is necessary to review a few definitions.

The support of an itemset is the number of transactions that contain the itemset. For example, the itemset {1, 5} has a support of 2 because it appears in three transactions from the database (t1 and t4). A generator is an itemset X such that there does not exist an itemset Y strictly included in X that has the same support. For example, itemset {1, 5} is a generator.

A high utility generator itemsets (HUG) is a high-utility itemset that is a generator.

For example, if we run HUG-Miner with a minimum utility of 20, we obtain 4 high-utility generator itemsets:

itemsets utility support
{2} 22 5 transactions
{1} 20 5 transactions
{2, 4} 30 2 transactions
{1 5} 24 2 transactions

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 20 $ or more, and that are minimal sets of items in common for a group of customers.

Input file format

The input file format of HUG-Miner is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of HUG-Miner is defined as follows. It is a text file, where each line represents a high utility generator itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUPPORT:" appears and is followed by the support of the itemset. Then, the keyword #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 #SUP: 5 #UTIL: 22
1 #SUP: 5 #UTIL: 20
4 2 #SUP: 2 #UTIL: 30
1 5 #SUP: 2 #UTIL: 24

For example, the third line indicates that the itemset {2, 4} has a support of 2 transactions and a utility of 30$. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The HUG-Miner algorithm was proposed in 2014 to discover only the high-utility itemsets that are generators. It is generally faster than discovering all high-utility itemsets. Thus, this algorithm can outperform algorithms such as FHM and HUI-Miner, who discover all high-utility itemsets.

Implementation details

This is the original implementation of HUG-Miner.

Where can I get more information about the HUG-Miner algorithm?

This is the reference of the article describing the HUG-Miner algorithm:

Fournier-Viger, P., Wu, C.W., Tseng, V.S. (2014). Novel Concise Representations of High Utility Itemsets using Generator Patterns. Proc. 10th Intern. Conference on Advanced Data Mining and Applications (ADMA 2014), Springer LNCS 8933, pp. 30-43.

Note that in this article, another algorithm called GHUI-Miner is also proposed. It is a different algorithm, which is also offered in SPMF.

Example 53 : Mining High-Utility Sequential Rules from a Sequence Database with utility information using the HUSRM Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUSRM" algorithm, (2) select the input file "DataBase_HUSRM.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 40, (5) set the minimum confidence to 0.7, (6) set the minimum antecedent size to 4, (7) set the maximum consequent size to 4, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUSRM DataBase_HUSRM.txt output.txt 40 0.7 4 4 in a folder containing spmf.jar and the example input file DataBase_HUSRM.txt.
  • If you are using the source code version of SPMF, launch the file "MainTest_HUSRM_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is HUSRM?

HUSRM (Zida et al, 2015) is the first algorithm for discovering high-utility sequential rules in a sequence database containing utility information.

An typical example of a sequence database with utility information is a database of customer transactions containing sequences of transactions performed by customers, where each transaction is a set of items annotated with the profit generated by the sale of items. The goal of high-utility sequential rule mining is to find rules of the form A -> B, meaning that if a customer buy items A, he will then buy items B with a high confidence, and this rule generate a high profit. Although, this algorithm is designed for the scenario of sequence of transactions, the task is general and could be applied to other types of data such as sequences of webpages visited by user on a website, where the sale profit is replaced by the time spent on webpages.

This is the original implementation of HUSRM.

Note that the problem of high-utility sequential rule mining is similar to high-utility sequential pattern mining. However, a key advantage of high-utility sequential rule mining is that discovered rules provide information about the probability that if some customers buy some items A, they will then buy other items B. High-utility sequential patterns do not consider the confidence that a pattern will be followed.

What is the input?

HUSRM takes as input a sequence database with utility information, a minimum utility threshold min_utility (a positive integer), a minimum confidence threshold (a double value in the [0,1] interval, a maximum antecedent size (a positive integer) and a maximm consequent size (a positive nteger).

Let's consider the following sequence database consisting of 4 sequences of transactions (s1,s2, s3, s4) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DataBase_HUSRM.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Sequence Sequence utility
s1 {1[1],2[4]},{3[10]},{6[9]},{7[2]},{5[1]} 27
s2 {1[1],4[12]},{3[20]},{2[4]},{5[1],7[2]} 40
s3 {1[1]},{2[4]},{6[9]},{5[1]} 15
s4 {1[3],2[4],3[5]},{6[3],7[1]} 16

Each line of the database is a sequence:

  • each sequence is an ordered list of transactions, such that transactions are enclosed by {} in this Example
  • each transaction contains a set of items represented by integers
  • each item is annotated with a utility value (e.g. sale profit), indicated between squared brackets
  • [ ].
  • the sum of the utilities (e.g. profit) of all items in the sequence is also indicated (the "sequence utility" column)

What are real-life examples of such a database? A typical example is a database containing sequences of customer transactions. Imagine that each sequence represents the transactions made by a customer. The first customer named "s1" bought items 1 and 2, and those items respectively generated a profit of 1$ and 4$. Then, the customer bought item 3 for 10$. Then, the customer bought item 6 for 9 $. Then, the customer bought items 7 for 2$. Then the customer bought item 5 for 1 $.

What is the output?

The output of HUSRM is the set of high utility sequential rules meeting the criteria specified by the user

A sequential rule X==>Y is a sequential relationship between two sets of items X and Y such that X and Y are disjoint, and that X is unordered and Y is unordered. The support of a rule X==>Y is the number of sequences that contains the items in X before the items in Y, divided by the number of sequences in the database. The confidence of a rule is the number of sequences that contains the items in X before the items in Y, divided by the number of sequences that contains X. For example, the rule {1,2,3} ==> {7} has a support of 3/5 because it appears in 3 out of 4 sequences in the database.

The utility (profit) of a rule is the total utility (profit) generated by the rule in the sequences where it appears. For example, the rule {1,2,3} ==> {7} appears in sequences s1,s2, and s4. In s1, the profit generated by that rule is 1$ + 4$ + 10$ + 2 $ = 17$. In s2, the profit generated by that rule is 1$ + 20$ + 4 + 2 $ = 27$. In s4, the profit generated by that rule is 3$ + 4$ + 5$ + 1$ =13$. Thus, the total utility of that rule in the database is 17$ + 27 $ + 13$ = 57 $

The HUSRM algorithm returns all high-utility sequential rules, that is each rule that meet the four following criteria:

  • the utility of the rule in the database is no less than a minimum utility threshold set by the user,
  • the confidence of the rule in the database is no less than a minimum confidence threshold set by the user,
  • the number of items in the antecedent (left side) of the rule contains no more than a maximum number of items specified by the user,
  • the number of items in the consequent (right side) of the rule contains no more than a maximum number of items specified by the user,

For example, if we run HUSRM with a minimum utility of 40 and minconf = 0.70 (70 %), and a maximum antecedent and consequent size of 4 items, we obtain 7 high-utility sequential rules:

rule confidence utility support
1,4 ==> 2,3,5,7 100 % 40 1 sequence(s)
1,3,4 ==> 2,5,7 100 % 40 1 sequence(s)
1,2,3,4 ==> 5,7 100 % 40 1 sequence(s)
1,2,3 ==> 7 100 % 57 3 sequence(s)
1,3 ==> 7 100 % 45 3 sequence(s)
2,3 ==> 7 100 % 52 3 sequence(s)
3 ==> 7 100 % 40 3 sequence(s)

If the database is a transaction database from a store, we could interpret these results as rules representing the purchasing behavior of customers, such that these rules have a high confidence and generate a high profit. For example, the rule {1,3} -> {7} means that all customers buying the items 1 and 3 always buy the item 7 thereafter (since the confidence is 100%) and that this rule has generated a profit of 57 $ and appear in three sequences.

Input file format

The input file format of HUSRM is defined as follows. It is a text file.

  • Each lines represents a sequence of transactions.
  • Each transaction is separated by the keyword -1.
  • A transaction is a list of items (positive integers) separated by single spaces and where each item is annotated with a generated sale profit indicated between square brackets [ ]. The sale profit is a positive integer.
  • In a transaction, it is assumed that items are sorted according to some order (eg. alphabetical order).
  • Each sequence ends by the keyword "-2". Then, it is followed by the keyword "SUtility:" followed by the sum of the utility (profit) of all items in that sequence.

For example, for the previous example, the input file is defined as follows:

1[1] 2[4] -1 3[10] -1 6[9] -1 7[2] -1 5[1] -1 -2 SUtility:27
1[1] 4[12] -1 3[20] -1 2[4] -1 5[1] 7[2] -1 -2 SUtility:40
1[1] -1 2[4] -1 6[9] -1 5[1] -1 -2 SUtility:15
1[3] 2[4] 3[5] -1 6[3] 7[1] -1 -2 SUtility:16

For examle, consider the first line. It means that the first customer nbought items 1 and 2, and those items respectively generated a profit of 1$ and 4$. Then, the customer bought item 3 for 10$. Then, the customer bought item 6 for 9 $. Then, the customer bought items 7 for 2$. Then the customer bought item 5 for 1 $. Thus, this customer has made 5 transaction. The total utility (profit) generated by that sequence of transaction is 1$ + 4$ + 10$ + 9$ + 2$ + 1$ = 27 $.

Output file format

The output file format of HUSRM is defined as follows. It is a text file, where each line represents a high utility sequential rule. On each line, the items of the left side of the rule (antecedent) are first listed. Each item is represented by an integer, followed by a ",". After, the keyword " ==>" appears. It is followed by the items in the right side of the rule (consequent), each separated by ",". Then, the keyword "#SUP" appears followed by the support of the rule. Then, the keyword "#CONF" appears followed by the confidence of the rule. Then, the keyword "#UTIL" appears followed by the utility of the rule.

1,4 ==> 2,3,5,7 #SUP: 1.0 #CONF: 1.0 #UTIL: 40.0
1,3,4 ==> 2,5,7 #SUP: 1.0 #CONF: 1.0 #UTIL: 40.0
1,2,3,4 ==> 5,7 #SUP: 1.0 #CONF: 1.0 #UTIL: 40.0
1,2,3 ==> 7 #SUP: 3.0 #CONF: 1.0 #UTIL: 57.0
1,3 ==> 7 #SUP: 3.0 #CONF: 1.0 #UTIL: 45.0
2,3 ==> 7 #SUP: 3.0 #CONF: 1.0 #UTIL: 52.0
3 ==> 7 #SUP: 3.0 #CONF: 1.0 #UTIL: 40.0

For example, the fourth line indicates that all customers buying the items 1, 2 and 3 will then buy item 7 with a confidence of 100%, and that this rule has generated a profit of 57 $ and appear in three sequences.

Performance

High utility sequential rulemining is a more difficult problem than sequential rule mining and sequential pattern mining. Therefore, high-utility sequential rule mining algorithms are generally slower than those types of algorithms. The HUSRM algorithm is the first algorithm for high-utility sequential rule mining.

Implementation details

This is the original implementation of HUSRM.

Where can I get more information about the HUSRM algorithm?

This is the article describing the HUSRM algorithm:

Zida, S., Fournier-Viger, P., Wu, C.-W., Lin, J. C. W., Tseng, V.S., (2015). Efficient Mining of High Utility Sequential Rules. Proc. 11th Intern. Conference on Machine Learning and Data Mining (MLDM 2015). Springer, LNAI 9166, pp. 157-171.

Example 54 : Mining Minimal High-Utility Itemsets from a transaction database with utility information using the MinFHM Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "MinFHM" algorithm, (2) select the input file "DB_utility.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 30 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run MinFHM DB_utility.txt output.txt 30 in a folder containing spmf.jar and the example input file DB_utility.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestMinFHM.java" in the package ca.pfv.SPMF.tests.

What is MinFHM?

MinFHM (Fournier-Viger et al., 2016) is an algorithm for discovering minimal high-utility itemsets in a transaction database containing utility information.

There has been quite a huge amount of work on the topic of high-utility itemset mining in recent years. High-utility itemset mining consists of finding sets of items that yield a high profit in a database of customer transaactions where the purchase quantities of items in transactions is indicated and each item has a unit profit. Several algorithms have been proposed for high-utility itemset mining. However, they may find a huge number of patterns. These patterns are often very long and often represent rare cases, as in real-life, few customers exactly buy the same large set of items. For marketing purpose, a retailer may be more interested in finding the smallest sets of items that generate a high profit, since it is easier to co-promote a small set of items targeted at many customers rather than a large set of items targeted at few customers. The MinFHM algorithm was designed to address this issues by discovering only the high-utility itemsets that are minimal.

A high-utility itemset is said to be minimal if it has no subset that is also a high-utility itemset. In terms of application to transaction database, the concept of minimal high-utility itemsets can be understood as the smallest sets of items that yield a high profit.. The concept of minimal high-utility itemset can also be understood as the opposite of the concept of maximal high-utility itemset proposed in other work.

This is the original implementation of MinFHM.

What is the input?

MinFHM takes as input a transaction database with utility information and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DB_utility.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Items Transaction utility Item utilities for this transaction
t1 3 5 1 2 4 6 30 1 3 5 10 6 5
t2 3 5 2 4 20 3 3 8 6
t3 3 1 4 8 1 5 2
t4 3 5 1 7 27 6 6 10 5
t5 3 5 2 7 11 2 3 4 2

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 3, 5, 1, 2, 4 and 6. The amount of money spent for each item is respectively 1 $, 3 $, 5 $, 10 $, 6 $ and 5 $. The total amount of money spent in this transaction is 1 + 3 + 5 + 10 + 6 + 5 = 30 $.

What is the output?

The output of MinFHM is the set of minimal high utility itemsets having a utility no less than a min_utility threshold (a positive integer) set by the user. To explain what is a minimal high-utility itemsets, it is necessary to review some definitions.

An itemset is an unordered set of distinct items. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {1 4} in transaction t1 is 5 + 6 = 11 and the utility of {1 4} in transaction t3 is 5 + 2 = 7. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {1 4} in the database is the utility of {1 4} in t1 plus the utility of {1 4} in t3, for a total of 11 + 7 = 18. A high utility itemset is an itemset such that its utility is no less than min_utility.

A minimal high utility itemsets (MinHUI) is a high-utility itemset that is has no subset that is a high-utility itemset

For example, if we run MinFHM with a minimum utility of 30, we obtain 2 minimal high-utility itemsets:

itemsets utility
{2, 4} 30
{2 5} 31

If the database is a transaction database from a store, we could interpret these results as all the smallest groups of items bought together that generated a profit of 30 $ or more (that are minimal).

Input file format

The input file format of MinFHM is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 5 1 2 4 6:30:1 3 5 10 6 5
3 5 2 4:20:3 3 8 6
3 1 4:8:1 5 2
3 5 1 7:27:6 6 10 5
3 5 2 7:11:2 3 4 2

Consider the first line. It means that the transaction {3, 5, 1, 2, 4, 6} has a total utility of 30 and that items 3, 5, 1, 2, 4 and 6 respectively have a utility of 1, 3, 5, 10, 6 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of MinFHM is defined as follows. It is a text file, where each line represents a high utility generator itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUPPORT:" appears and is followed by the support of the itemset. Then, the keyword #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

2 #SUP: 5 #UTIL: 22
1 #SUP: 5 #UTIL: 20
4 2 #SUP: 2 #UTIL: 30
1 5 #SUP: 2 #UTIL: 24

For example, the third line indicates that the itemset {2, 4} has a support of 2 transactions and a utility of 30$. The following lines follows the same format.

Performance

High utility itemset mining is a more difficult problem than frequent itemset mining. Therefore, high-utility itemset mining algorithms are generally slower than frequent itemset mining algorithms. The MinFHM algorithm was proposed in 2016 to discover only the high-utility itemsets that are minimal. It was found that MinFHM can be orders of magitude faster than algorithms such as FHM for mining all high-utility itemsets.

Implementation details

This is the original implementation of the MinFHM algorithm

Where can I get more information about the MinFHM algorithm?

This is the reference of the article describing the MinFHM algorithm:

Fournier-Viger, P., Lin, C.W., Wu, C.-W., Tseng, V. S., Faghihi, U. (2016). Mining Minimal High-Utility Itemsets. Proc. 27th International Conference on Database and Expert Systems Applications (DEXA 2016). Springer, LNCS, 13 pages, to appear

Example 55 : Mining Skyline High-Utility Itemsets in a transaction database with utility information using the SkyMine Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "SkyMine" algorithm, (2) select the input file "SkyMineTransaction.txt", (3) set the output file name (e.g. "output.txt"), (4) set the name of the second input file to "SkyMineItemUtilities.txt" and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run SkyMine SkyMineTransaction.txt output.txt SkyMineItemUtilities.txt in a folder containing spmf.jar and the example input files SkyMineTransaction.txt and SkyMineItemUtilities.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestSkyMine_saveToFile" in the package ca.pfv.SPMF.tests.

What is SkyMine?

SkyMine (Goyal et al, 2015) is an algorithm for discovering skyline high-utility itemsets in a transaction database containing utility information.

This is the original implementation of SkyMine.

What is the input?

SkyMine takes as input a transaction database with purchase quantities, a table indicating the utility of items, and a minimum utility threshold min_utility (a positive integer). Let's consider the following database consisting of 5 transactions (t1,t2...t5) and 9 items (1, 2, 3, 4, 5, 6, 7, 8, 9). This database is provided in the text file "SkyMineTransaction.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Items Item purchase quantities for this transaction
t1 1 3 4 8 1 1 1 1
t2 1 3 5 7 2 6 2 5
t3 1 2 3 4 5 6 1 2 1 6 1 5
t4 2 3 5 7 2 2 1 2
t5 1 3 4 9 1 1 1 1

Each line of the database is:

  • a set of items (the first column of the table),
  • the purchase quantities of these items in this transaction (the second column of the table),

For example, the second line of the database indicates that in the second transaction, the items 1, 3, 5, and 7 were purchased respectively with quantities of 2, 6, 2, and 5.

Moreover, another table must be provided to indicate the unit profit of each item (how much profit is generated by the sale of one unit of each item). For example, consider the utility table provided in the file "SkyMineItemUtilities.txt (below). The first line indicates that each unit sold of item 1 yield a profit of 5$.

Item Utility (unit profit)
1 5
2 2
3 1
4 2
5 3
6 1
7 1
8 1
9 25

What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 1, 3, 4 and 8. The purchase quantities of each item is respectively 1, 1, 1, and 1. The total amount of money spent in this transaction is (1*5)+(3*1)+(4*2)+(8*1)= 24 $.

 

What is the output?

The output of SkyMine is the set of skyline high utility itemsets. To explain what is a skyline high-utility itemsets, it is necessary to review some definitions.

An itemset is an unordered set of distinct items. The utility of an item in a transaction is the product of its purchase quantity in the transaction by its unit profit. For example, the utility of item 3 in transaction t2 is (6*1)- 6 $. The utility of an itemset in a transaction is the sum of the utility of its items in the transaction. For example, the utility of the itemset {5 7} in transaction t2 is (2*3)+(5*1)=12$ and the utility of {5, 7} in transaction t4 is (1*3)+(2*1)=5. The utility of an itemset in a database is the sum of its utility in all transactions where it appears. For example, the utility of {5 7} in the database is the utility of {5 7} in t4 plus the utility of {5 7} in t5, for a total of 12 + 5= 17. The utility of an itemset X is denoted as u(X). Thus u({5 7})= 17$

The support of an itemset is the number of transactions that contains the itemset. For example, the support of the itemset {5 7} is sup({5 7}) = 2 transactions because it appears in transactions t4 and t5.

An itemset X is said to be dominating another itemset Y, if and only if, sup(X) ≥ sup(Y ) and u(X) > u(Y ), or, sup(X) > sup(Y ) and u(X) ≥ u(Y ).

A skyline high utility itemset is an itemset that is not dominated by another itemset in the transaction database.

For example, if we run SkyMine, we obtain 3 skyline high-utility itemsets:

itemsets utility
{3} 14
{1, 3} 34
{2, 3, 4, 5} 40

If the database is a transaction database from a store, we could interpret these results as all the itemsets that are dominating the other itemsets in terms of selling frequencies and utilty.

Input file format

The input file format of the transaction file of Skymine is defined as follows. It is a text file. Each lines represents a transaction. Each transaction is a list of items separated by single spaces. Each item is a positive integer followed by ":" and its purchase quantity in the transaction. Note that it is assume that items on each line are ordered according to some total order such as the alphabetical order. For example, for the previous example, the input file SkyMineTransactions.txt is defined as follows:

1:1 3:1 4:1 8:1
1:2 3:6 5:2 7:5
1:1 2:2 3:1 4:6 5:1 6:5
2:4 3:3 4:3 5:1
2:2 3:2 5:1 7:2
1:1 3:1 4:1 9:1

For example, the second line indicates that the items 1, 3, 5 and 7 respectively have a purchase quantity of 2, 6, 2 and 5 in that transaction.

The input format of the second file, indicating the utility (unit profit) of each item, is defined as follows. Each line is an item, followed by a space, followed by the unit profit of the item. For example, consider the content of the file "SkyMineItemUtilities.txt", shown below. The first line indicates that the item 1 has a unit profit of 5$. The other lines follow the same format.

1 5
2 2
3 1
4 2
5 3
6 1
7 1
8 1
9 25

Output file format

The output file format of SkyMine is defined as follows. It is a text file, where each line represents a skyline high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. Then, the keyword #UTIL: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.

3 #UTIL: 14
1 3 #UTIL: 34
2 3 4 5 #UTIL: 40

For example, the third line indicates that the itemset {2, 3, 4, 5} has a utility of 40$. The other lines follows the same format.

Performance

SkyMine is the original algorithm for mining Skyline high-utility itemets.

Where can I get more information about the algorithm?

This is the reference of the article describing the algorithm:

Goyal, V., Sureka, A., & Patel, D. (2015). Efficient Skyline Itemsets Mining. In Proceedings of the Eighth International C* Conference on Computer Science & Software Engineering (pp. 119-124). ACM.

Example 56 : Mining High-Utility Sequential Patterns from a Sequence Database with utility information using the USPAN Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "USPAN" algorithm, (2) select the input file "DataBase_HUSRM.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 35, (5) set the maximum pattern length to 4, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run USpan DataBase_HUSRM.txt output.txt 35 4 in a folder containing spmf.jar and the example input file DataBase_HUSRM.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestUSpan.java" in the package ca.pfv.SPMF.tests.

What is USpan?

USpan (Zida et al, 2012) is a famous algorithm for discovering high-utility sequential patterns in a sequence database containing utility information.

An typical example of a sequence database with utility information is a database of customer transactions containing sequences of transactions performed by customers, where each transaction is a set of items annotated with the profit generated by the sale of items. The goal of high-utility sequential rule mining is to find patterns of the form A, B, C, meaning that several customers have bought items A, followed by buying item B, followed by buying item C, and that this pattern generated a high profit. Although, this algorithm is designed for the scenario of sequence of transactions, the task is general and could be applied to other types of data such as sequences of webpages visited by user on a website, where the sale profit is replaced by the time spent on webpages.

A limitation of the problem of high-utility sequential pattern mining is that patterns are only found based on the profit that they generate but there is no measure of the confidence that these patterns will be followed. For example, a pattern A,B,C may have a high utility but most customers may still buy items A,B without buying C. The alternative that proposed a solution to this problem is high-utility sequential rule mining, which discover rules of the form A -> BC with a confidence (conditional probability). The algorithm HUSRM also offered in SPMF finds the high-utility sequential rules.

What is the input?

USPAN takes as input a sequence database with utility information, a minimum utility threshold min_utility (a positive integer), an optionally, a maximum pattern length parameter (a positive integer) indicating the maximum number of items that a pattern should contani.

Let's consider the following sequence database consisting of 4 sequences of transactions (s1,s2, s3, s4) and 7 items (1, 2, 3, 4, 5, 6, 7). This database is provided in the text file "DataBase_HUSRM.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.


Sequence Sequence utility
s1 {1[1],2[4]},{3[10]},{6[9]},{7[2]},{5[1]} 27
s2 {1[1],4[12]},{3[20]},{2[4]},{5[1],7[2]} 40
s3 {1[1]},{2[4]},{6[9]},{5[1]} 15
s4 {1[3],2[4],3[5]},{6[3],7[1]} 16

Each line of the database is a sequence:

  • each sequence is an ordered list of transactions, such that transactions are enclosed by {} in this Example
  • each transaction contains a set of items represented by integers
  • each item is annotated with a utility value (e.g. sale profit), indicated between squared brackets
  • [ ].
  • the sum of the utilities (e.g. profit) of all items in the sequence is also indicated (the "sequence utility" column)

Note that this representation of the input database is not exactly the same as in the paper about USpan. However, it is equivalent.

What are real-life examples of such a database? A typical example is a database containing sequences of customer transactions. Imagine that each sequence represents the transactions made by a customer. The first customer named "s1" bought items 1 and 2, and those items respectively generated a profit of 1$ and 4$. Then, the customer bought item 3 for 10$. Then, the customer bought item 6 for 9 $. Then, the customer bought items 7 for 2$. Then the customer bought item 5 for 1 $.

What is the output?

The output of USPAN is the set of high utility sequential patterns meeting the criteria specified by the user

A sequential pattern is a sequence of itemsets X1, X2, ... Xk, where X1, X2... Xk are itemsets (sets of items). A sequential pattern is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 is a subset of Yi1, X2 s a subset of Yi2, ... Xk s a subset of Yik.

The utility (profit) of a sequential pattern is the sum of the maximum utility (profit) generated by the pattern in each sequences where it appears. For example, the rule (3)(7) appears in sequences s1,s2, and s4. In s1, the profit generated by that patern is 10 + 2 = 12 $. In s2, the profit generated by that pattern is 20 + 2 = 22 $. In s4, the profit generated by that pattern is 5+1 = 6$. Thus, the total utility of that rule in the database is 12 + 22 + 6 = 40 $.

The USPAN algorithm returns all high-utility sequential patterns, such that each pattern the two following criteria:

  • the utility of the rule in the database is no less than a minimum utility threshold set by the user,
  • the confidence of the rule in the database is no less than a minimum confidence threshold set by the user,
  • the number of items in the antecedent (left side) of the rule contains no more than a maximum number of items specified by the user,
  • the number of items in the consequent (right side) of the rule contains no more than a maximum number of items specified by the user,

For example, if we run USPANwith a minimum utility of 35 and a maximum pattern length of 4 items, we obtain 9 high-utility sequential patterns:

rule utility
(1, 4), (3) (2) 37
(1, 4) (3) (7) 35
(1) (3) (7) 36
(3) 35
(3) (7) 40
(4) (3) (2) 36
(4) (3) (2) (5) 37
(4) (3) (2) (7) 38
(4) (3) (2) (5, 7) 35

If the database is a transaction database from a store, we could interpret these results as rules representing the purchasing behavior of customers, such that these rules have a high confidence and generate a high profit. For example, the rule {1,3} -> {7} means that all customers buying the items 1 and 3 always buy the item 7 thereafter (since the confidence is 100%) and that this rule has generated a profit of 57 $ and appear in three sequences.

Input file format

The input file format of USPAN is defined as follows. It is a text file.

  • Each lines represents a sequence of transactions.
  • Each transaction is separated by the keyword -1.
  • A transaction is a list of items (positive integers) separated by single spaces and where each item is annotated with a generated sale profit indicated between square brackets [ ]. The sale profit is a positive integer.
  • In a transaction, it is assumed that items are sorted according to some order (eg. alphabetical order).
  • Each sequence ends by the keyword "-2". Then, it is followed by the keyword "SUtility:" followed by the sum of the utility (profit) of all items in that sequence.

For example, for the previous example, the input file is defined as follows:

1[1] 2[4] -1 3[10] -1 6[9] -1 7[2] -1 5[1] -1 -2 SUtility:27
1[1] 4[12] -1 3[20] -1 2[4] -1 5[1] 7[2] -1 -2 SUtility:40
1[1] -1 2[4] -1 6[9] -1 5[1] -1 -2 SUtility:15
1[3] 2[4] 3[5] -1 6[3] 7[1] -1 -2 SUtility:16

For examle, consider the first line. It means that the first customer nbought items 1 and 2, and those items respectively generated a profit of 1$ and 4$. Then, the customer bought item 3 for 10$. Then, the customer bought item 6 for 9 $. Then, the customer bought items 7 for 2$. Then the customer bought item 5 for 1 $. Thus, this customer has made 5 transaction. The total utility (profit) generated by that sequence of transaction is 1$ + 4$ + 10$ + 9$ + 2$ + 1$ = 27 $.

Output file format

The output file format of USPAN is defined as follows. It is a text file, where each line represents a high utility sequential pattern. Each line, first indicate the sequential patterns which is a list of itemsets. Each itemset is represented by a list of positive integers. Each itemset is separated by a -1. Then, the keyword "#UTIL" appears followed by the utility of the sequential pattern.

1 4 -1 3 -1 2 -1 #UTIL: 37
1 4 -1 3 -1 7 -1 #UTIL: 35
1 -1 3 -1 7 -1 #UTIL: 36
3 -1 #UTIL: 35
3 -1 7 -1 #UTIL: 40
4 -1 3 -1 2 -1 #UTIL: 36
4 -1 3 -1 2 -1 5 -1 #UTIL: 37
4 -1 3 -1 2 -1 7 -1 #UTIL: 38
4 -1 3 -1 5 7 -1 #UTIL: 35

For example, the first line represents the pattern of buying items 1 and 4 together, then buying item 3, then buying item 2. This pattern has a total utility of 37, meaning that it generated a 37 $ profit. The other lines follow the same format.

Performance

High utility sequential pattern mining is a more difficult problem than sequential pattern mining. Therefore, high-utility sequential pattern mining algorithms are generally slower than sequential pattern mining algorithm. For this reason, it is wise to use the optional maximum pattern length constraint when using USpan, to reduce the number of patterns found, and thus the size of the search space.

It is also worth noting that in the USpan paper they do not compare the performance of their algorithm with previous algorithms for high-utility sequential pattern mining.

Where can I get more information about the USPAN algorithm?

This is the article describing the USPAN algorithm:

Yin, Junfu, Zhigang Zheng, and Longbing Cao. "USpan: an efficient algorithm for mining high utility sequential patterns." Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2012.

Example # 57: Mining High-Utility Itemsets based on Particle Swarm Optimization with the HUIM-BPSO algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the HUIM-BPSO algorithm, (2) select the input file contextHUIM.txt, (3) set the minutil parmeter to 40, (4) set the output file name (e.g. "output.txt") and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command: java -jar spmf.jar run HUIM-BPSO contextHUIM.txt output.txt 40 in a folder containing spmf.jar and the example input file contextHUIM.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHUIM_BPSO.java" in the package ca.pfv.SPMF.tests.

What is HUIM-BPSO?

HUIM-BPSO is an algorithm for discovering high utility itemsets (HUIs) which have utility value no less than the minimum utility threshold in a transaction database. The HUIM-BPSO algorithm discovers HUIs using binary particle swarm optimization (BPSO).

 What is the input?

HUIM-BPSO takes as input a transaction database with utility information. Let's consider the following database consisting of 7 transactions (t1,t2, ..., t7) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file "contextHUIM.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Items

Transaction utility

Item utilities for this transaction

t1

2 3 4

9

2 2 5

t2

1 2 3 4 5

18

4 2 3 5 4

t3

1 3 4

11

4 2 5

t4

3 4 5

11

2 5 4

t5

1 2 4 5

22

5 4 5 8

t6

1 2 3 4

17

3 8 1 5

t7

4 5

9

5 4

Each line of the database is:

    • a set of items (the first column of the table),
    • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
    • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.
What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 2, 3 and 4. The amount of money spent for each item is respectively 2 $, 2 $ and 5 $. The total amount of money spent in this transaction is 2 + 2 + 5 = 9 $.

What is the output?

The output of HUIM-BPSO is the set of high utility itemsets. An itemset X in a database D is a high-utility itemset (HUI) if and only if its utility is no less than the minimum utility threshold (a positive integer). A high utility itemset is an itemset such that its utility is no less than min_utility

For example, if we run HUIM-BPSO and set minimum utility threshold as 40, we obtain 2 high utility itemsets.


itemsets

utility

{4,5}

40

{1,2,4}

41

 

Input file format

The input file format of high utility itemsets is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

2 3 4:9:2 2 5
1 2 3 4 5:18:4 2 3 5 4
1 3 4:11:4 2 5
3 4 5:11:2 5 4
1 2 4 5:22:5 4 5 8
1 2 3 4:17:3 8 1 5
4 5:9:5 4

Consider the first line. It means that the transaction {2, 3, 4} has a total utility of 9 and that items 2, 3 and 4 respectively have a utility of 2, 2 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of high utility itemsets is defined as follows. It is a text file, each following line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTILITY: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.
4 5 #UTIL: 40
1 2 4 #UTIL: 41

For example, the first line indicates that the itemset {4, 5} is a high utility itemset which has utility equals to 41. The following lines follows the same format.

Implementation details

The version implemented here contains all the optimizations described in the paper proposing HUIM-BPSO. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the HUIM-BPSO algorithm?

This is the reference of the article describing the HUIM-BPSO algorithm:

Jerry Chun-Wei Lin, Lu Yang, Philippe Fournier-Viger, Ming-Thai Wu, Tzung-Pei Hong, Leon Shyue-Liang Wang, and Justin Zhan, “Mining High-Utility Itemsets based on Particle Swarm Optimization,” Engineering Applications of Artificial Intelligence, Vol. 55, pp: 320-330, 2016.

Example # 58: Mining High-Utility Itemsets based on Particle Swarm Optimization with the HUIM-BPSO-tree algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUIM-BPSO-tree" algorithm, (2) select the input file "contextHUIM.txt", (3) set the output file name (e.g. "output.txt"), (4) set the minutil threshold to 40, and (4) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUIM-BPSO-tree contextHUIM.txt output.txt 40
    in a folder containing spmf.jar and the example input contextHUIM.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHUIM_BPSO_tree.java" in the package ca.pfv.SPMF.tests.

What is HUIM-BPSO-tree?

HUIM-BPSO-tree is an algorithm for discovering high utility itemsets (HUIs) which have utility value no less than the minimum utility threshold in a transaction database. The HUIM-BPSO-tree algorithm discovers HUIs based on binary particle swarm optimization (BPSO) algorithm and designed OR/NOR-tree structure to avoid combinations, which can improve the efficiency to discovering HUIs.

What is the input?

HUIM-BPSO-tree takes as input a transaction database with utility information. Let's consider the following database consisting of 7 transactions (t1,t2, ..., t7) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file "contextHUIM.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Items

Transaction utility

Item utilities for this transaction

t1

2 3 4

9

2 2 5

t2

1 2 3 4 5

18

4 2 3 5 4

t3

1 3 4

11

4 2 5

t4

3 4 5

11

2 5 4

t5

1 2 4 5

22

5 4 5 8

t6

1 2 3 4

17

3 8 1 5

t7

4 5

9

5 4

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.
What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 2, 3 and 4. The amount of money spent for each item is respectively 2 $, 2 $ and 5 $. The total amount of money spent in this transaction is 2 + 2 + 5 = 9 $.

What is the output?

The output of HUIM-BPSO-tree is the set of high utility itemsets. An itemset X in a database D is a high-utility itemset (HUI) iff its utility is no less than the minimum utility threshold. For example, if we run HUIM-BPSO-tree and set minimum utility threshold as 40, we obtain 2 high utility itemsets.


itemsets

utility

{4,5}

40

{1,2,4}

41

Input file format

The input file format of high utility itemsets is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:
2 3 4:9:2 2 5
1 2 3 4 5:18:4 2 3 5 4
1 3 4:11:4 2 5
3 4 5:11:2 5 4
1 2 4 5:22:5 4 5 8
1 2 3 4:17:3 8 1 5
4 5:9:5 4

Consider the first line. It means that the transaction {2, 3, 4} has a total utility of 9 and that items 2, 3 and 4 respectively have a utility of 2, 2 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of high utility itemsets is defined as follows. It is a text file, each following line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTILITY: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.
4 5 #UTIL: 40
1 2 4 #UTIL: 41

For example, the first line indicates that the itemset {4, 5} is a high utility itemset which has utility equals to 41. The following lines follows the same format.

Implementation details

The version implemented here contains all the optimizations described in the paper proposing HUIM-BPSO-tree. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the HUIM-BPSO-tree algorithm?

This is the reference of the article describing the HUIM-BPSO-tree algorithm:

Jerry Chun-Wei Lin, Lu Yang, Philippe Fournier-Viger, Tzung-Pei Hong, and Miroslav Voznak, “A Binary PSO Approach to Mine High-Utility Itemsets,” Soft Computing, pp: 1-19, 2016.

Example # 59: Discovery of High Utility Itemsets Using a Genetic Algorithm with the HUIM-GA algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUIM-GA" algorithm, (2) select the input file "contextHUIM.txt", (3) set the output file name (e.g. "output.txt"), (4) set the minutil parameter to 40, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUIM-GA contextHUIM.txt output.txt 40
    in a folder containing spmf.jar and the example input contextHUIM.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHUIM_GA.java" in the package ca.pfv.SPMF.tests.

What is HUIM-GA?

HUIM-GA is an algorithm for discovering high utility itemsets (HUIs) which have utility value no less than the minimum utility threshold in a transaction database. The HUIM-GA algorithm discovers HUIs using a genetic algorithm (GA).

 What is the input?

HUIM-GA takes as input a transaction database with utility information. Let's consider the following database consisting of 7 transactions (t1,t2, ..., t7) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file "contextHUIM.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Items

Transaction utility

Item utilities for this transaction

t1

2 3 4

9

2 2 5

t2

1 2 3 4 5

18

4 2 3 5 4

t3

1 3 4

11

4 2 5

t4

3 4 5

11

2 5 4

t5

1 2 4 5

22

5 4 5 8

t6

1 2 3 4

17

3 8 1 5

t7

4 5

9

5 4

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.
What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 2, 3 and 4. The amount of money spent for each item is respectively 2 $, 2 $ and 5 $. The total amount of money spent in this transaction is 2 + 2 + 5 = 9 $.

What is the output?

The output of HUIM-GA is the set of high utility itemsets. An itemset X in a database D is a high-utility itemset (HUI) if and only if its utility is no less than the minimum utility threshold. For example, if we run HUIM-GA and set the minimum utility threshold to 40, we obtain 2 high utility itemsets.


itemsets

utility

{4,5}

40

{1,2,4}

41

Input file format

The input file format of high utility itemsets is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:
2 3 4:9:2 2 5
1 2 3 4 5:18:4 2 3 5 4
1 3 4:11:4 2 5
3 4 5:11:2 5 4
1 2 4 5:22:5 4 5 8
1 2 3 4:17:3 8 1 5
4 5:9:5 4

Consider the first line. It means that the transaction {2, 3, 4} has a total utility of 9 and that items 2, 3 and 4 respectively have a utility of 2, 2 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of high utility itemsets is defined as follows. It is a text file, each following line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTILITY: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.
4 5 #UTIL: 40
1 2 4 #UTIL: 41

For example, the first line indicates that the itemset {4, 5} is a high utility itemset which has utility equals to 41. The following lines follows the same format.

Implementation details

The version implemented here contains all the optimizations described in the paper proposing HUIM-GA. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the HUIM-GA algorithm?

This is the reference of the article describing the HUIM-GA algorithm:

Kannimuthu S, Premalatha K, “Discovery of High Utility Itemsets Using Genetic Algorithm with Ranked Mutation,” Applied Artificial Intelligence,2014, 28(4): 337-359.

Example # 60: Discovery of High Utility Itemsets Using a Genetic Algorithm with the HUIM-GA-tree algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HUIM-GA-tree " algorithm, (2) select the input file "contextHUIM.txt", (3) set the output file name (e.g. "output.txt"), (4) set the minutil parameter to 40, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HUIM-GA-tree contextHUIM.txt output.txt 40
    in a folder containing spmf.jar and the example input contextHUIM.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHUIM_GA_tree.java" in the package ca.pfv.SPMF.tests.

What is HUIM-GA-tree?

HUIM-GA-tree is an algorithm for discovering high utility itemsets (HUIs) which have utility value no less than the minimum utility threshold in a transaction database. The HUIM-GA-tree algorithm discovers HUIs using a genetic algorithm (GA) and a OR/NOR-tree structure.

 What is the input?

HUIM-GA-tree takes as input a transaction database with utility information. Let's consider the following database consisting of 7 transactions (t1,t2, ..., t7) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file "contextHUIM.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Items

Transaction utility

Item utilities for this transaction

t1

2 3 4

9

2 2 5

t2

1 2 3 4 5

18

4 2 3 5 4

t3

1 3 4

11

4 2 5

t4

3 4 5

11

2 5 4

t5

1 2 4 5

22

5 4 5 8

t6

1 2 3 4

17

3 8 1 5

t7

4 5

9

5 4

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.
What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 2, 3 and 4. The amount of money spent for each item is respectively 2 $, 2 $ and 5 $. The total amount of money spent in this transaction is 2 + 2 + 5 = 9 $.

What is the output?

The output of HUIM-GA-tree is the set of high utility itemsets. An itemset X in a database D is a high-utility itemset (HUI) iff its utility is no less than the minimum utility threshold. For example, if we run HUIM-GA-tree and set the minimum utility threshold to 40, we obtain 2 high utility itemsets.


itemsets

utility

{4,5}

40

{1,2,4}

41

Input file format

The input file format of high utility itemsets is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:
2 3 4:9:2 2 5
1 2 3 4 5:18:4 2 3 5 4
1 3 4:11:4 2 5
3 4 5:11:2 5 4
1 2 4 5:22:5 4 5 8
1 2 3 4:17:3 8 1 5
4 5:9:5 4

Consider the first line. It means that the transaction {2, 3, 4} has a total utility of 9 and that items 2, 3 and 4 respectively have a utility of 2, 2 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of high utility itemsets is defined as follows. It is a text file, each following line represents a high utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #UTILITY: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.
4 5 #UTIL: 40
1 2 4 #UTIL: 41

For example, the first line indicates that the itemset {4, 5} is a high utility itemset which has utility equals to 41. The following lines follows the same format.

Implementation details

The version implemented here contains all the optimizations described in the paper proposing HUIM-GA-tree. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the HUIM-GA-tree algorithm?

The HUIM-GA-tree algorithm is a combination of the HUIM-GA algorithm and the OR/NOR-tree structure. The reference of the article describing the original HUIM-GA algorithm:

Kannimuthu S, Premalatha K, “Discovery of High Utility Itemsets Using Genetic Algorithm with Ranked Mutation,” Applied Artificial Intelligence,2014, 28(4): 337-359.

The HUIM-GA-tree algorithm with OR/NOR-tree structure is described in:

Jerry Chun-Wei Lin, Lu Yang, Philippe Fournier-Viger, Tzung-Pei Hong, and Miroslav Voznak, “A Binary PSO Approach to Mine High-Utility Itemsets,” Soft Computing, pp: 1-19, 2016.

 

Example 61 : Mining All Association Rules

How to run this example?

  • If you are using the graphical interface, (1) choose the "FPGrowth_association_rules" algorithm, (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50 %, minconf= 60% (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FPGrowth_association_rules contextIGB.txt output.txt 50% 60% in a folder containing spmf.jar and the example input file contextIGB.txt.
  • If you are using the source code version of SPMF, launch this file "MainTestAllAssociationRules_FPGrowth_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

It is an algorithm for discovering all association rules in a transaction database, following the two steps approach proposed by Agrawal & Srikant (1993). The first step is to discover frequent itemsets. The second step is to generate rules by using the frequent itemsets. The main difference with Agrawal & Srikant in this implementation is that FPGrowth is used to generate frequent itemsets instead of Apriori because FPGrowth is more efficient.

What is the input?

The input is a transaction database (aka binary context) and two thresholds named minsup (a value between 0 and 1) and minconf (a value between 0 and 1).

A transaction database is a set of transactions. Each transaction is a set of distinct items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

What is the output?

The output of an association rule mining algorithm is a set of association rules respecting the user-specified minsup and minconf thresholds. To explain how this algorithm works, it is necessary to review some definitions. An association rule X==>Y is a relationship between two itemsets (sets of items) X and Y such that the intersection of X and Y is empty. The support of a rule is the number of transactions that contains X∪Y. The confidence of a rule is the number of transactions that contains X∪Y divided by the number of transactions that contain X.

If we apply an association rule mining algorithm, it will return all the rules having a support and confidence respectively no less than minsup and minconf.

For example, by applying the algorithm with minsup = 0.5 (50%), minconf = 0.6 (60%), we obtains 55 associations rules (run the example in the SPMF distribution to see the result).

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

Consider the first line. It means that the first transaction is the itemset {1, 2, 4 and 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by an integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer. Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is a few lines from the output file for this example:

1 ==> 2 4 5 #SUP: 3 #CONF: 0,75
5 ==> 1 2 4 #SUP: 3 #CONF: 0,6
4 ==> 1 2 5 #SUP: 3 #CONF: 0,75

For example, the first line indicates that the association rule {1} --> {2, 4, 5} has a support of 3 transactions and a confidence of 75 %. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Implementation details

Association rule mining is traditionally performed in two steps : (1) mining frequent itemset and (2) generating association rules by using frequent itemsets. In this implementation, we use the FPGrowth algorithm for Step 1 because it is very efficient. For Step 2, we use the algorithm that was proposed by Agrawal & Srikant (1994).

Note that in SPMF, we offer also the alternative of choosing Apriori instead of FPGrowth for Step1. This is called the "Apriori_association_rules" algorithm in the graphical user interface or command line interface.

Lastly, note that we offer also the alternative of choosing CFPGrowth++ instead of FPGrowth for Step1. This is called the "CFPGrowth++_association_rules" algorithm in the graphical user interface or command line interface. CFPGrowth++ allows to use multiple minimum support threshold instead of a single minsup thresholds so the input and output are slightly different (see the example about CFPGrowth++ for more details about this algorithm).

Where can I get more information about this algorithm?

The following technical report published in 1994 describes how to generate association rules from frequent itemsets (Step 2):

R. Agrawal and R. Srikant. Fast algorithms for mining association rules in large databases. Research Report RJ 9839, IBM Almaden Research Center, San Jose, California, June 1994.

You can also read chapter 6 of the book "introduction to data mining" which provide a nice and easy to understand introduction to how to discover frequent itemsets and generate association rules.

The following article describes the FPGrowth algorithm for mining frequent itemsets:

Jiawei Han, Jian Pei, Yiwen Yin, Runying Mao: Mining Frequent Patterns without Candidate Generation: A Frequent-Pattern Tree Approach. Data Min. Knowl. Discov. 8(1): 53-87 (2004)

Example # 62: Mining Skyline Frequent-Utility Patterns using the SFUPMinerUemax algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "SFUPMinerUemax" algorithm, (2) select the input file "contextHUIM.txt", (3) set the output file name (e.g. "output.txt") and (4) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run SFUPMinerUemax contextHUIM.txt output.txt in a folder containing spmf.jar and the example input contextHUIM.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestSFUPMinerUemax.java" in the package ca.pfv.SPMF.tests.

What is SFUPMinerUemax?

SFUPMinerUemax is an algorithm for discovering skyline frequent-utility patterns (SFUPs) in a transaction database containing utility information. The SFUPMinerUemax algorithm discovers SFUPs by exploring a utility-list structure using a depth-first search. An efficient pruning strategy is also developed to prune unpromising candidates early and thus reduce the search space

 What is the input?

SFUPMinerUemax takes as input a transaction database with utility information. Let's consider the following database consisting of 7 transactions (t1,t2, ..., t7) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file "contextHUIM.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Items

Transaction utility

Item utilities for this transaction

t1

2 3 4

9

2 2 5

t2

1 2 3 4 5

18

4 2 3 5 4

t3

1 3 4

11

4 2 5

t4

3 4 5

11

2 5 4

t5

1 2 4 5

22

5 4 5 8

t6

1 2 3 4

17

3 8 1 5

t7

4 5

9

5 4

Each line of the database is:

  • a set of items (the first column of the table),
  • the sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • the utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.
What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 2, 3 and 4. The amount of money spent for each item is respectively 2 $, 2 $ and 5 $. The total amount of money spent in this transaction is 2 + 2 + 5 = 9 $.

What is the output?

The output of SFUPMinerUemax is the set of skyline frequent-utility patterns. An itemset X in a database D is a skyline frequent-utility patterns (SFUP) iff it is not dominated by any other itemset in the database by considering both the frequent and utility factors. An itemset X dominates another itemset Y in D, iff f(X) >= f(Y) and u(X) >= u(Y). For example, if we run SFUPMinerUemax, we obtain 3 skyline frequent-utility patterns.


itemsets

support

utility

{2,1,4}

3

41

{5,4}

4

40

{4}

7

35

Input file format

The input file format of SFUPMinerUemax is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:
2 3 4:9:2 2 5
1 2 3 4 5:18:4 2 3 5 4
1 3 4:11:4 2 5
3 4 5:11:2 5 4
1 2 4 5:22:5 4 5 8
1 2 3 4:17:3 8 1 5
4 5:9:5 4

Consider the first line. It means that the transaction {2, 3, 4} has a total utility of 9 and that items 2, 3 and 4 respectively have a utility of 2, 2 and 5 in this transaction. The following lines follow the same format.

Output file format

The output file format of the algorithm is defined as follows. It is a text file, the first line record the count of skyline frequent-utility patterns. And each following line represents a skyline frequent-utility pattern. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword "#SUP:" appears, which is followed by a integer value indicating the support of that itemset and the keyword " #UTILITY: " appears and is followed by the utility of the itemset. For example, we show below the output file for this example.
Total skyline frequent-utility itemset: 3
2 1 4 #SUP:3 #UTILITY:41
5 4 #SUP:4 #UTILITY:40
4 #SUP:7 #UTILITY:35

For example, the first line indicates that there are 3 skyline frequent-utility patterns in the example. The second line indicates that the itemset {2, 1, 4} is a skyline frequent-utility itemset which has support equals to 3 and utility equals to 41. The following lines follows the same format.

Implementation details

The version implemented here contains all the optimizations described in the paper proposing SFUPMinerUemax. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the SFUPMinerUemax algorithm?

This is the reference of the article describing the SFUPMinerUemax algorithm:

Jerry Chun-Wei Lin, Lu Yang, Philippe Fournier-Viger, Siddharth Dawar, Vikram Goyal, Ashish Sureka, and Bay Vo, “A More Efficient Algorithm to Mine Skyline Frequent-Utility Patterns,” International Conference on Genetic and Evolutionary Computing, 2016. (ICGEC 2016)

Example 63 : Mining High Average-Utility Itemsets in a Transaction Database with Utility Information using the HAUI-Miner Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HAUI-Miner" algorithm, (2) select the input file "contextHAUIMiner.txt", (3) set the output file name (e.g. "output.txt") (4) set the minimum utility to 24 and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HAUI-Miner contextHAUIMiner.txt output.txt 24 in a folder containing spmf.jar and the example input file contextHAUIMiner.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHAUIMiner_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is HAUI-Miner?

HAUI-Miner is an algorithm for discovering high average-utility itemsets (HAUIs) in a transaction database containing utility information. The HAUI-Miner algorithm discovers HAUIs by exploring a set-enumeration tree using a depth-first search. An efficient pruning strategy is also developed to prune unpromising candidates early and thus reduce the search space

What is the input?

HAUI-Miner takes as input a transaction database with utility information and a minimum utility threshold minAUtility (a positive integer). Let's consider the following database consisting of six transactions (t1, t2, ... , t6) and 6 items (1, 2, 3, 4, 5, 6). This database is provided in the text file "contextHAUIMiner.txt" in the package ca.pfv.spmf.tests of the SPMF distribution.

Items

Transaction utility

Item utilities for this transaction

t1

1 2 3 4 6

32

5 6 6 9 6

t2

2 3 5

16

2 6 8

t3

1 3 4 5

22

10 2 6 4

t4

1 2 3 4 6

28

5 9 6 6 2

t5

1 2 3 4 5

37

15 9 6 3 4

t6

3 4 5

15

8 3 4

Each line of the database is:

  • A set of items (the first column of the table),
  • The sum of the utilities (e.g. profit) of these items in this transaction (the second column of the table),
  • The utility of each item for this transaction (e.g. profit generated by this item for this transaction)(the third column of the table).

Note that the value in the second column for each line is the sum of the values in the third column.
What are real-life examples of such a database? There are several applications in real life. One application is a customer transaction database. Imagine that each transaction represents the items purchased by a customer. The first customer named "t1" bought items 1, 2, 3, 4 and 6. The amount of money spent for each item is respectively 5 $, 6 $, 6 $, 9 $ and 6 $. The total amount of money spent in this transaction is 5 + 6 + 6 + 9 + 6 = 32 $.

What is the output?

The output of HAUI-Miner is the set of high average-utility itemsets having an average-utility no less than a minAUtility threshold (a positive integer) set by the user. Average utility measure estimates the utility of an itemset by considering its length. It is defined as the sum of the utilities of the itemset in transactions where it appears, divided by the number of items that it contains. For example, the average-utility of {2, 3, 5} in the database is the utility of {2, 3, 5} in t2 plus the utility of {2, 3, 5} in t5, for a total of 16 + 19 = 35, divide by 3, equals 11.6. A high average-utility itemset is an itemset such that its utility is no less than minAUtility. For example, if we run HAUI-Miner with a minimum utility of 24, we obtain 10 high average-utility itemsets.

itemsets

average-utility

{1}

35

{2}

26

{3}

34

{4}

27

{1 2}

24

{1 3}

27

{1, 4}

29

{2, 3}

25

{3, 4}

27

{1, 3, 4}

26

If the database is a transaction database from a store, we could interpret these results as all the groups of items bought together that generated a profit of 24 $ or more, when divided by the number of items.

Input file format

The input file format of HAUI-Miner is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

1 2 3 4 6:32:5 6 6 9 6
2 3 5:16:2 6 8
1 3 4 5:22:10 2 6 4
1 2 3 4 6:28:5 9 6 6 2
1 2 3 4 5:37:15 9 6 3 4
3 4 5:15:8 3 4

Consider the first line. It means that the transaction {1, 2, 3, 4, 6} has a total utility of 32 and that items 1, 2, 3, 4, and 6 respectively have a utility of 5, 6, 6, 9 and 6 in this transaction. The following lines follow the same format.

Output file format

The output file format of HAUI-Miner is defined as follows. It is a text file, where each line represents a high average-utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #AUTIL: " appears and is followed by the average utility of the itemset. For example, we show below the output file for this example.

2  #AUTIL: 26
2 1  #AUTIL: 24
2 3  #AUTIL: 25
1  #AUTIL: 35
1 4  #AUTIL: 29
1 4 3  #AUTIL: 26
1 3  #AUTIL: 27
4  #AUTIL: 27
4 3  #AUTIL: 27
3  #AUTIL: 34

For example, the first line indicates that the itemset {2} has an average-utility of 26. The following lines follows the same format.

Implementation details

The version implemented here contains all the optimizations described in the paper proposing HAUI-Miner. Note that the input format is not exactly the same as described in the original article. But it is equivalent.

Where can I get more information about the HAUI-Miner algorithm?

This is the reference of the article describing the HAUI-Miner algorithm:

Jerry Chun-Wei Lin, Ting Li, Philippe Fournier-Viger, Tzung-Pei Hong, Justin Zhan, and Miroslav Voznak. An Efficient Algorithm to Mine High Average-Utility Itemsets[J]. Advanced Engineering Informatics, 2016, 30(2):233-243.

Example 64 : Mining High Average-Utility Itemsets with Multiple Thresholds in a Transaction Database using the HAUI-MMAU Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HAUI-MMAU" algorithm, (2) select the input file "contextHAUIMMAU.txt", (3) set the output file name (e.g. "output.txt") (4) set the GLMAU to 0 and the MAUI file to "MAU_Utility.txt" and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run HAUI-MMAU contextHAUIMMAU.txt output.txt 0 MAU_Utility.txt in a folder containing spmf.jar and the example input file contextHAUIMMAU.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHAUI_MMAU_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is HAUI-MMAU?

HAUI-MMAU is an algorithm for mining average-utility itemsets by using multiple minimum average-utility thresholds. Unlike some other algorithms such as HAUI-Miner, the HAUI-MMAU algorithms allows to set a different threshold for each item, rather than a single threshold to evaluate all items. Setting multiple thresholds is useful because it allows to set lower minimum average-utility thresholds for low profit items. Therefore, it allows discovering high average-utility itemsets containing low profit items.

What is the input?

The input of HAUI-MMAU is a transaction database and a list of minimum average-utility thresholds indicating the minimum average-utility threshold for each item.
A transaction database is a set of transactions, where each transaction is a list of distinct items (symbols). For example, let's consider the following transaction database. It consists of 5 transactions (t1, t2, ..., t5) and 8 items (1, 2, 3, 4, 5, 6). For instance, transaction t1 is the set of items {2, 3, 4, 5}. This database is provided in the file "contextHAUIMMAU.txt" of the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction.


Transaction ID

Items

Utilities

t1

2 3 4 5

14 2 6 4

t2

2 3 4

8 3 6

t3

1 4

10 2

t4

1 3 6

5 6 4

t5

2 3 4 6

4 3 2 2

The list of minimum average-utility threshold is stored in a text file that is read as input by the algorithm. This is provided in the file "MAU_Utility.txt":


item

minimum average-utility threshold

1

5

2

2

3

1

4

2

5

4

6

1

This file indicated for example that the average-utility threshold to be used for item 1 is 5.

What is the output?

The output of HAUI-MMAU is the set of all high average-utility itemsets contained in the database.

What is a high average-utility itemset? The average-utility of an itemset is the sum of the average-utilities of transactions containing the itemset. An itemset is a high average-utility itemset if its average-utility is higher or equal to the smallest minimum average-utility threshold among the minimum average-utility thresholds of all its items. For example, the itemset {1, 4} is high average-utility because it appears in transactions (t3) and its average-utility (= 6)is higher than the average minimum average-utility item 1, item 4, which is (5+2)/2 = 3.5.

Why HAUI-MMAU is useful? It is useful because it permits setting lower minimum average-utility thresholds for low profit items. Therefore, it allows discovering high average-utility itemsets containing low profit items.
If we run HAUI-MMAU on the previous transaction database with the MAU_Utility.txt file previously described, we get the following result, where each line represents an itemsets followed by ":" and then its absolute average-utility and minimum average-utility count:

1  #AUTIL: 15 #mau: 5.0
2  #AUTIL: 26 #mau: 2.0
3  #AUTIL: 14 #mau: 1.0
4  #AUTIL: 16 #mau: 2.0
5  #AUTIL: 4 #mau: 4.0
6  #AUTIL: 6 #mau: 1.0
3 6  #AUTIL: 7 #mau: 1.0
3 2  #AUTIL: 17 #mau: 1.5
3 4  #AUTIL: 11 #mau: 1.5
3 5  #AUTIL: 3 #mau: 2.5
3 1  #AUTIL: 5 #mau: 3.0
6 2  #AUTIL: 3 #mau: 1.5
6 4  #AUTIL: 2 #mau: 1.5
6 1  #AUTIL: 4 #mau: 3.0
2 4  #AUTIL: 20 #mau: 2.0
2 5  #AUTIL: 9 #mau: 3.0
4 5  #AUTIL: 5 #mau: 3.0
4 1  #AUTIL: 6 #mau: 3.5
3 6 2  #AUTIL: 3 #mau: 1.3333334
3 6 4  #AUTIL: 2 #mau: 1.3333334
3 6 1  #AUTIL: 5 #mau: 2.3333333
3 2 4  #AUTIL: 16 #mau: 1.6666666
3 2 5  #AUTIL: 6 #mau: 2.3333333
3 4 5  #AUTIL: 4 #mau: 2.3333333
6 2 4  #AUTIL: 2 #mau: 1.6666666
2 4 5  #AUTIL: 8 #mau: 2.6666667
3 6 2 4  #AUTIL: 2 #mau: 1.5
3 2 4 5  #AUTIL: 6 #mau: 2.25

For example, the first line represent the itemset “3, 2” has average-utility 17 and its minimum average-utility count is 1.5. The other lines follows the same format.

Input file format

HAUI-MMAU takes two files as input, defined as follows.

The first file (e.g. contextHAUIMMAU.txt) is a text file containing transactions. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the transaction utility (an integer).
  • Third, the symbol ":" appears and is followed by the utility of each item in this transaction (an integer), separated by single spaces.

2 3 4 5:26:14 2 6 4
2 3 4:17:8 3 6
1 4:12:10 2
1 3 6:15:5 6 4
2 3 4 6:11:4 3 2 2

Consider the first line. It means that the first transaction is the itemset {2, 3, 4, 5} with utilities {14, 2, 6, 4}. The following lines follow the same format.

The second file is a text file (e.g. MAU_Utility.txt) which provides the minimum average-utility to be used for each item. Each line indicate the minimum average-utility for an item and consists of two integer values separated by a single space. The first value is the item. The second value is the minimum average-utility value to be used for this item. For example, here is the file used in this example. The first line indicate that for item "1" and the minimum average-utility to be used is 1 (one transaction). The other lines follow the same format.
1 5
2 2
3 1
4 2
5 4
6 1

Output file format

The output file format of HAUI-Miner is defined as follows. It is a text file, where each line represents a high average-utility itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by a single space. After, all the items, the keyword " #AUTIL: " appears and is followed by the average-utility of the itemset. For example, we show below the output file for this example.

1  #AUTIL: 15 #mau: 5.0
2  #AUTIL: 26 #mau: 2.0
3  #AUTIL: 14 #mau: 1.0
4  #AUTIL: 16 #mau: 2.0
5  #AUTIL: 4 #mau: 4.0
6  #AUTIL: 6 #mau: 1.0
3 6  #AUTIL: 7 #mau: 1.0
3 2  #AUTIL: 17 #mau: 1.5
3 4  #AUTIL: 11 #mau: 1.5
3 5  #AUTIL: 3 #mau: 2.5
3 1  #AUTIL: 5 #mau: 3.0
6 2  #AUTIL: 3 #mau: 1.5
6 4  #AUTIL: 2 #mau: 1.5
6 1  #AUTIL: 4 #mau: 3.0
2 4  #AUTIL: 20 #mau: 2.0
2 5  #AUTIL: 9 #mau: 3.0
4 5  #AUTIL: 5 #mau: 3.0
4 1  #AUTIL: 6 #mau: 3.5
3 6 2  #AUTIL: 3 #mau: 1.3333334
3 6 4  #AUTIL: 2 #mau: 1.3333334
3 6 1  #AUTIL: 5 #mau: 2.3333333
3 2 4  #AUTIL: 16 #mau: 1.6666666
3 2 5  #AUTIL: 6 #mau: 2.3333333
3 4 5  #AUTIL: 4 #mau: 2.3333333
6 2 4  #AUTIL: 2 #mau: 1.6666666
2 4 5  #AUTIL: 8 #mau: 2.6666667
3 6 2 4  #AUTIL: 2 #mau: 1.5
3 2 4 5  #AUTIL: 6 #mau: 2.25

For example, the last line indicates that the itemset {3 2 4 5} has average utility of 6 which is larger than its minimum average-utility threshold 2.25. The other lines follows the same format.

Implementation details

This is the original implementation of the algorithm.

Where can I get more information about the HAUI-MMAU algorithm?

This is the reference of the article describing the HAUI-MMAU algorithm:

Jerry Chun-Wei Lin, Ting Li, Philippe Fournier-Viger, Tzung-Pei Hong, and Ja-Hwung Su. Efficient Mining of High Average-Utility Itemsets with Multiple Minimum Thresholds[C]//Proceedings of the Industrial Conference on Data Mining, 2016:14-28.

Example # 65: Mining Fuzzy Frequent Itemsets in a quantitative transaction database using the FFI-Miner algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FFI-Miner" algorithm, (2) select the input file "contextFFIMiner.txt", (3) set the output file name (e.g. "output.txt"), (4) set the minsup parameter to 2, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command: 
    java -jar spmf.jar run FFI-Miner contextFFIMiner.txt output.txt 2 in a folder containing spmf.jar and the example input file contextFFIMiner.txt .
  •  If you are using the source code version of SPMF, launch the file "MainTestFFIMiner_saveToFile.java" in the package ca.pfv. SPMF.tests.

What is FFI-Miner?

FFI-Miner is an algorithm for mining fuzzy frequent itemsets in quantitative transactional database. In simple words, a quantitative transactional database is a database where items have quantities.

 What is the input?

FFI-Miner takes as input (1) a transaction database with quantity information and a minimum support threshold minSupport (a positive integer). Let's consider the following database consisting of 8 transactions (t1, t2, ..., t8) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file “contextFFIMiner.txt ”in the package ca.pfv.spmf.tests of the SPMF distribution. Moreover, consider the membership function, shown below, which defines three ranges (low, medium, high).


Transaction ID

Items

Quantities

t1

1 3 4 5

5 10 2 9

t2

1 2 3

8 2 3

t3

2 3

3 9

t4

1 2 3 5

5 3 10 3

t5

1 3 4

7 9 3

t6

2 3 4

2 8 3

t7

1 2 3

5 2 5

t8

1 3 4 5

3 10 2 2

Fuzzy membership function

Why FFI-Miner is useful?

In real-life situations, it is difficult to handle quantitative databases using crisp sets. Fuzzy-set theory is useful to handle quantitative databases. Based on the fuzzy-set theory, the fuzzy frequent itemset mining algorithm FFI-Miner was proposed. It relies on a designed fuzzy-list structure to discover fuzzy itemsets. Compared to previous works, FFI-Miner has excellent performance for the discovery of fuzzy itemsets.

Input file format

The input file format of FFI-Miner is defined as follows. It is a text file. Each line represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the sum of the quantities in that transaction.
  • Third, the symbol ":" appears and is followed by the quantity of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

1 3 4 5:26:5 10 2 9
1 2 3:13:8 2 3
2 3:12:3 9
1 2 3 5:21:5 3 10 3
1 3 4:19:7 9 3
2 3 4:13:2 8 3
1 2 3:12:5 2 5
1 3 4 5:19:3 10 2 2

Consider the first line. It means that the transaction {1, 3, 4, 5} has a total quantity of 26 and that items1, 3, 4 and 5 respectively have a quantity of 5, 10, 2 and 9 in this transaction. The following lines follow the same format.

Output file format

The output file format of FFI-Miner is defined as follows. It is a text file, where each line represents a fuzzy frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by “.” and either the letter “L”, “M” or “H”. These letters are used to indicate if an item is in the low-range, medium-range or high range in terms of the fuzzy membership function. After, all the items, the keyword "#FLV:" appears, which is followed by a float value indicating the fuzzy value of that itemset.
1.M #FVL: 4.2
1.M 2.L #FVL: 2.0
1.M 3.H #FVL: 2.6000001
2.L #FVL: 3.6
4.L #FVL: 2.8
4.L 3.H #FVL: 2.6000001
3.H #FVL: 4.0000005

For example, the first line indicates that the itemset 1 has a fuzzy value of 4.2 and is in the medium range of the fuzzy membership function. The other lines follow the same format.

Performance

FFI-Miner is a very efficient algorithm. It uses a designed fuzzy-list structure to identify unpromising candidates early, and thus speed up the discovery of fuzzy itemsets.

Where can I get more information about the algorithm?

This is the reference of the article describing the FFI-Miner algorithm:

Jerry Chun-Wei Lin, Ting Li, Philippe Fournier-Viger, and Tzung-Pei Hong. A Fast Algorithm for Mining Fuzzy Frequent Itemsets[J]. Journal of Intelligent & Fuzzy Systems, 2015, 29(6):2373-2379.

Example # 66: Mining Multiple Fuzzy Frequent Itemsets in a quantitative transaction database using the MFFI-Miner algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "MFFI-Miner" algorithm, (2) select the input file "contextMFFIMiner.txt", (3) set the output file name (e.g. "output.txt"), (4) set the minsup parameter to 2, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command: 
    java -jar spmf.jar run MFFI-Miner contextMFFIMiner.txt output.txt 2 in a folder containing spmf.jar and the example input file contextMFFIMiner.txt .
  •  If you are using the source code version of SPMF, launch the file "MainTestMFFIMiner_saveToFile.java" in the package ca.pfv. SPMF.tests.

What is MFFI-Miner?

MFFI-Miner is an algorithm for mining fuzzy frequent itemsets in quantitative transactional database. In simple words, a quantitative transactional database is a database where items have quantities.

 What is the input?

MFFI-Miner takes as input (1) a transaction database with quantity information and a minimum support threshold minSupport (a positive integer). Let's consider the following database consisting of 8 transactions (t1, t2, ..., t8) and 5 items (1, 2, 3, 4, 5). This database is provided in the text file “contextMFFIMiner.txt ” in the package ca.pfv.spmf.tests of the SPMF distribution.

Moreover, consider the membership function, shown below, which defines three ranges (low, medium, high).

Transaction ID

Items

Quantitates

t1

3 4 5

3 2 1

t2

2 3 4

1 2 1

t3

2 3 5

3 3 1

t4

1 3 4

3 5 3

t5

1 2 3 4

1 1 2 1

t6

2 4 5

1 1 2

t7

1 2 4 5

4 3 5 3

t8

2 3 4

1 2 1

Fuzzy membership function

Why MFFI-Miner is useful?

Previous work on fuzzy frequent itemset mining used the maximum scalar cardinality to mine fuzzy frequent itemsets (FFIs), in which at most, only one linguistic term was used to represent the item in the databases. Although this process can reduce the amount of computation for mining FFIs, the discovered information may be invalid or incomplete. A gradual data-reduction approach (GDF) for mining multiple fuzzy frequent itemsets (MFFIs).The tree-based algorithm UBMFFP-tree suffered from building a huge tree structure. It mines FFIs with multiple fuzzy regions based on an Apriori-like mechanism. The MFFI-Miner algorithm efficiently mines MFFIs without candidate generation based on the designed fuzzy-list structure. This approach can be used to reduce the amount of computation and avoid using a generate-candidate-and-test approach with a level-wise exploration of the search space

Input file format

The input file format of MFFI-Miner is defined as follows. It is a text file. Each lines represents a transaction. Each line is composed of three sections, as follows.

  • First, the items contained in the transaction are listed. An item is represented by a positive integer. Each item is separated from the next item by a single space. It is assumed that all items within a same transaction (line) are sorted according to a total order (e.g. ascending order) and that no item can appear twice within the same transaction.
  • Second, the symbol ":" appears and is followed by the sum of the item quantities (an integer).
  • Third, the symbol ":" appears and is followed by the quantity of each item in this transaction (an integer), separated by single spaces.

For example, for the previous example, the input file is defined as follows:

3 4 5:6:3 2 1
2 3 4:4:1 2 1
2 3 5:7:3 3 1
1 3 4:11:3 5 3
1 2 3 4:5:1 1 2 1
2 4 5:4:1 1 2
1 2 4 5:15:4 3 5 3
2 3 4:4:1 2 1

Consider the first line. It means that the transaction {3, 4, 5} has a total quantity of 6 and that items 3, 4 and 5 respectively have a quantity of 3, 2 and 1 in this transaction. The following lines follow the same format.

Output file format

The output file format of MFFI-Miner is defined as follows. It is a text file, where each line represents a frequent itemset. On each line, the items of the itemset are first listed. Each item is represented by an integer, followed by the letter "L", "M" or "H", and a single space. The letters L, M and H indicate if an item is in the Low, Medium or High Range of the fuzzy membership function, respectively. After, all the items, the keyword "#SUP:" appears, which is followed by a integer value indicating the support of that itemset.
3.H #SUP: 2.0
2.L #SUP: 4.0
2.L 3.M #SUP: 2.0
2.L 3.M 4.L #SUP: 2.0
2.L 4.L #SUP: 4.0
3.M #SUP: 3.3333335
3.M 4.L #SUP: 2.5000002
5.L #SUP: 2.5
4.L #SUP: 4.5

For example, the first line indicates that the itemset 3 in the high range of the fuzzy membership function (H) has a fuzzy value of 2.0. The other lines follows the same format.

Performance

MFFI-Miner is a very efficient algorithm. It uses a designed fuzzy-list structure to identify unpromising candidates early, and thus speed up the discovery of fuzzy itemsets.

Where can I get more information about the algorithm?

This is the article describing the MFFI-Miner algorithm:

Jerry Chun-Wei Lin, Ting Li, Philippe Fournier-Viger, Tzung-Pei Hong, Jimmy     Ming-Thai Wu, and Justin Zhan. Efficient Mining of Multiple Fuzzy Frequent Itemsets[J]. International Journal of Fuzzy Systems, 2016:1-9.

Some other related papers:

T. P. Hong, G. C. Lan, Y. H. Lin, and S. T. Pan, An effective gradual data-reduction strategy for fuzzy itemset mining, International Journal of Fuzzy Systems, Vol. 15(2), pp.170-181, 2013. (GDF)

J. C. W. Lin, T. P. Hong, T. C. Lin, and S. T. Pan, An UBMFFP tree for mining multiple fuzzy frequent itemsets, International Journal of Uncertainty, Fuzziness and Knowledge- Based Systems, Vol. 23(6), pp. 861-879, 2015. (UBMFFP-tree)

Example 67 : Mining All Association Rules with the Lift Measure

How to run this example?

  • If you are using the graphical interface, (1) choose the "FPGrowth_association_rules_with_lift" algorithm , (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50 %, minconf= 90% and minlift = 1 (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FPGrowth_association_rules_with_lift contextIGB.txt output.txt 50% 90% 1 in a folder containing spmf.jar and the example input file contextIGB.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestAllAssociationRules_FPGrowth_version_with_lift.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

This is a variation of the algorithm for mining all association rules from a transaction database, described in the previous example.

Traditionally, association rule mining is performed by using two interestingness measures named the support and confidence to evaluate rules. In this example, we show how to use another popular measure that is called the lift or interest.

What is the input?

The input is a transaction database (aka binary context) and three thresholds named minsup (a value between 0 and 1), minconf (a value between 0 and 1) and minlift (a value between -infinity to +infinity).

A transaction database is a set of transactions. Each transaction is a set of distinct items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

What is the output ?

The output of this algorithm is a set of all the association rules that have a support, confidence and lift respectively higher than minsup, minconf and minlift.

The lift of a rule X-->Y is calculated as lift(X-->Y) = ( (sup(X U Y)/ N) / (sup(X)/ N*sup(Y)/ N ), where

  • N is the number of transactions in the transaction database,
  • sup(X∪Y) is the number of transactions containing X and Y,
  • sup(X) is the number of transactions containing X
  • sup(Y) is the number of transactions containing Y.

The confidence of a rule X-->Y is calculated as conf(X-->Y) = sup(X U Y) / (sup(X)).

The support of a rule X -->Y is defined as sup(X-->Y) = sup(X∪Y) / N

By applying the algorithm with minsup = 0.5, minconf= 0.9 and minlift = 1 on the previous database, we obtains 18 associations rules:

  rule 0:   4  ==> 2      support :  0.66 (4/6) confidence :  1.0  lift :  1.0
rule 1: 3 ==> 2 support : 0.66 (4/6) confidence : 1.0 lift : 1.0
rule 2: 1 ==> 5 support : 0.66 (4/6) confidence : 1.0 lift : 1.2
rule 3: 1 ==> 2 support : 0.66 (4/6) confidence : 1.0 lift : 1.0
rule 4: 5 ==> 2 support : 0.833(5/6) confidence : 1.0 lift : 1.0
rule 5: 4 5 ==> 2 support : 0.5 (3/6) confidence : 1.0 lift : 1.0
rule 6: 1 4 ==> 5 support : 0.5 (3/6) confidence : 1.0 lift : 1.2
rule 7: 4 5 ==> 1 support : 0.5 (3/6) confidence : 1.0 lift : 1.5
rule 8: 1 4 ==> 2 support : 0.5 (3/6) confidence : 1.0 lift : 1.0
rule 9: 3 5 ==> 2 support : 0.5 (3/6) confidence : 1.0 lift : 1.0
rule 10: 1 5 ==> 2 support : 0.66 (4/6) confidence : 1.0 lift : 1.0
rule 11: 1 2 ==> 5 support : 0.66 (4/6) confidence : 1.0 lift : 1.2
rule 12: 1 ==> 2 5 support : 0.66 (4/6) confidence : 1.0 lift : 1.2
rule 13: 1 4 5 ==> 2 support : 0.5 (3/6) confidence : 1.0 lift : 1.0
rule 14: 1 2 4 ==> 5 support : 0.5 (3/6) confidence : 1.0 lift : 1.2
rule 15: 2 4 5 ==> 1 support : 0.5 (3/6) confidence : 1.0 lift : 1.5
rule 16: 4 5 ==> 1 2 support : 0.5 (3/6) confidence : 1.0 lift : 1.5
rule 17: 1 4 ==> 2 5 support : 0.5 (3/6) confidence : 1.0 lift : 1.5

How to interpret the results?

For an association rule X ==> Y, if the lift is equal to 1, it means that X and Y are independent. If the lift is higher than 1, it means that X and Y are positively correlated. If the lift is lower than 1, it means that X and Y are negatively correlated. For example, if we consider the rule {1, 4} ==> {2, 5}, it has a lift of 1.5, which means that the occurrence of the itemset {1, 4} is positively correlated with the occurrence of {2, 5}.

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

Consider the first line. It means that the first transaction is the itemset {1, 2, 4 and 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by an integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer. Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). Then, the keyword " #LIFT: " appears followed by the lift of the rule represented by a double value (a value between -infinity to +infinity). For example, here is a few lines from the output file for this example:

1 ==> 2 4 5 #SUP: 3 #CONF: 0,75 #LIFT: 1,5
5 ==> 1 2 4 #SUP: 3 #CONF: 0,6 #LIFT: 1,2

For example, the first line indicates that the association rule {1} --> {2, 4, 5} has a support of 3 transactions, a confidence of 75 % and a lift of 1.5 indicating a positive correlation (when the value is higher than 1). The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Implementation details

In the source code version of SPMF, there are two versions of this algorithm. The first version saves the result into memory ("MainTestAllAssociationRules_FPGrowth_wifthLift"). The second one saves the result to a file ("MainTestAllAssociationRules_FPGrowth_saveToFile_wifthLift").

Note that we offer also the alternative of choosing CFPGrowth++ instead of FPGrowth. This is called the "CFPGrowth++_association_rules_lift" algorithm in the graphical user interface or command line interface. CFPGrowth++ allows to use multiple minimum support thresholds instead of a single minsup threshold so the input and output are slightly different (see the example about CFPGrowth++ for more details about this algorithm).

Where can I get more information about this algorithm?

The following technical report published in 1994 describes how to generate association rules from frequent itemsets (Step 2):

R. Agrawal and R. Srikant. Fast algorithms for mining association rules in large databases. Research Report RJ 9839, IBM Almaden Research Center, San Jose, California, June 1994.

You can also read chapter 6 of the book "introduction to data mining" which provide a nice and easy to understand introduction to how to discover frequent itemsets and generate association rules, and also describes the advantages of using the lift measure.

The following article describes the FPGrowth algorithm for mining frequent itemsets:

Jiawei Han, Jian Pei, Yiwen Yin, Runying Mao: Mining Frequent Patterns without Candidate Generation: A Frequent-Pattern Tree Approach. Data Min. Knowl. Discov. 8(1): 53-87 (2004)

Example 68 : Mining All Association Rules using the GCD algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "GCD_association_rules" algorithm, (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50 %, minconf= 60%, maxcomb = 3, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run GCD_association_rules contextIGB.txt output.txt 50% 60% 3 in a folder containing spmf.jar and the example input file contextIGB.txt.
  • If you are using the source code version of SPMF, launch this file "MainTestGCDAssociationRules.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

This algorithm finds the association rules in a given transaction database or sequence of transactions/events, using GCD calculations for prime numbers.It is an original algorithm implemented by Ahmed El-Serafy and Hazem El-Raffiee.

What is the input?

The input is a transaction database (aka binary context) and three thresholds named minsup (a value between 0 and 1), minconf (a value between 0 and 1), and maxcomb (a positive integer).

A transaction database is a set of transactions. Each transaction is a set of distinct items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

What is the output?

The output of an association rule mining algorithm is a set of association rules respecting the user-specified minsup and minconf thresholds. To explain how this algorithm works, it is necessary to review some definitions. An association rule X==>Y is a relationship between two itemsets (sets of items) X and Y such that the intersection of X and Y is empty. The support of a rule is the number of transactions that contains X∪Y. The confidence of a rule is the number of transactions that contains X∪Y divided by the number of transactions that contain X.

If we apply an association rule mining algorithm, it will return all the rules having a support and confidence respectively no less than minsup and minconf.

For example, by applying the algorithm with minsup = 0.5 (50%), minconf = 0.6 (60%), and maxcomb = 3, we obtain 56 associations rules (run the example in the SPMF distribution to see the result).

Now let's explain the "maxcomb" parameter taken by the GCD algorithm. This parameter is used by the algorithm when finding the GCD (greatest common divisors) between two transactions. For example, consider 385, which comes from the multiplication of (5, 7 and 11), this actually means that (5), (7), (11), (5, 7), (5, 11), (7, 11), (5, 7, 11) are all common combinations between these two transactions. For larger GCD's, calculating all combinations grows exponentially in both time and memory. Hence, we introduced this parameter, to limit the maximum combinations' length generated from a single GCD. Although increasing this number might seem to provide more accurate results, the experiments showed that larger association rules occur at lower support (less important to the user). Hence, setting this parameter to values from 1 to 4 produces reasonable results.

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

Consider the first line. It means that the first transaction is the itemset {1, 2, 4 and 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by an integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer. Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is a few lines from the output file for this example:

1 ==> 2 4 5 #SUP: 3 #CONF: 0,75
5 ==> 1 2 4 #SUP: 3 #CONF: 0,6
4 ==> 1 2 5 #SUP: 3 #CONF: 0,75

For example, the first line indicates that the association rule {1} --> {2, 4, 5} has a support of 3 transactions and a confidence of 75 %. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

 

Where can I get more information about this algorithm?

The GCD Association Rules algorithm is an original algorithm. More information about it can be obtained from the bitbucket repository dedicated to this algorithm: https://bitbucket.org/aelserafy/gcd-association-rules

Example 69 : Mining the IGB basis of Association Rules

How to run this example?

  • If you are using the graphical interface, (1) choose the "IGB" algorithm , (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50 %, minconf= 61% (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run IGB contextIGB.txt output.txt 50% 61% in a folder containing spmf.jar and the example input file contextIGB.txt.
  • To run this example with the source code version of SPMF, launch the file "MainTestIGB_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

This algorithm mines a subset of all association rules that is called IGB association rules (Informative and Generic Basis of Association Rules) from a transaction database.

To discover the IGB association rules, this algorithm performs two steps: (1) first it discovers Closed itemsets and their associated generators by applying the Zart algorithm. Then (2), association rules are generated by using closed itemsets and generators.

What is the input?

The input is a transaction database and two thresholds named minsup (a value between 0 and 1) and minconf (a value between 0 and 1) .

A transaction database is a set of transactions. Each transaction is a set of distinct items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt of the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

What is the output?

The output is the IGB basis of association rules. It is a compact set of association rules that is both informative and generic. To explain what is the IGB basis of association rules, it is necessary to review some definitions. An itemset is a group of items. The support of an itemset is the number of times that it appears in the database divided by the total number of transactions in the database. For example, the itemset {1 3} has a support of 33 % because it appears in 2 out of 6 transactions from the database.

An association rule X--> Y is an association between two itemsets X and Y that are disjoint. The support of an association rule is the number of transactions that contains X and Y divided by the total number of transactions. The confidence of an association rule is the number of transactions that contains X and Y divided by the number of transactions that contains X.

A closed itemset is an itemset that is strictly included in no itemset having the same support. An itemset Y is the closure of an itemset X if Y is a closed itemset, X is a subset of Y and X and Y have the same support. A generator Y of a closed itemset X is an itemset such that (1) it has the same support as X and (2) it does not have a subset having the same support.

The IGB set of association rules is the set of association rules of the form X ==> Y - X, where X is a minimal generator of Y, Y is a closed itemset having a support higher or equal to minsup, and the confidence of the rule is higher or equal to minconf.

For example, by applying the IGB algorithm on the transaction database previously described with minsup = 0.50 and minconf= 0.61, we obtain the following set of association rules:

Rule Support Confidence
1 ==> 2, 4, 5 0.50 0.75
4 ==> 1, 2, 5 0.50 0.75
3 ==> 2, 5 0.50 0.75
{} ==> 2, 3 0.66 0.66
{} ==> 1, 2, 5 0.66 0.66
{} ==> 2, 4 0.66 0.66
{} ==> 2, 5 0.83 0.83
{} ==> 2 1 1

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

Consider the first line. It means that the first transaction is the itemset {1, 2, 4 and 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by an integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer. Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is a few lines from the output file for this example:

1 ==> 2 4 5 #SUP: 0,5 #CONF: 0,75
3 ==> 2 5 #SUP: 3 #CONF: 0.75

For example, the first line indicates that the association rule {1} --> {2, 4, 5} has a support of 3 transactions and a confidence of 75 %. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Where can I get more information about IGB association rules?

This article described IGB rules:

G. Gasmi, S. Ben Yahia, E. Mephu Nguifo, Y. Slimani: IGB: A New Informative Generic Base of Association Rules. PAKDD 2005: 81-90

Example 70 : Mining Perfectly Sporadic Association Rules

How to run this example?

  • If you are using the graphical interface, (1) choose the "Sporadic_association_rules" algorithm , (2) select the input file "contextInverse.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 1 %, maxsup = 60%, minconf= 60% (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Sporadic_association_rules contextInverse.txt output.txt 1% 60% 60% in a folder containing spmf.jar and the example input file contextInverse.txt.
  • To run this example with the source code version of SPMF, launch the file "MainTestAllPerfectlySporadicRules.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

This is an algorithm for mining perfectly sporadic association rules. The algorithm first uses AprioriInverse to generate perfectly rare itemsets. Then, it uses these itemsets to generate the association rules.

What is the input?

The input of this algorithm is a transaction database and three thresholds named minusp, maxsup and minconf. A transaction database is a set of transactions. A transaction is a set of distinct items (symbols), assumed to be sorted in lexical order. For example, the following transactions database contains 5 transactions (t1,t2...t5) and 5 items (1,2,3,4,5). This database is provided in the file "contextInverse.txt" of the SPMF distribution:

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3}
t5 {1, 2, 4, 5}

What is the output?

The output is the set of perfectly sporadic association rules respecting the minconf (a value in [0,1]), minsup (a value in [0,1]) and maxsup (a value in [0,1]) parameters.

To explain what it a perfectly sporadic association rule, we need to review some definitions. An itemset is an unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset divided by the total number of transactions. For example, the itemset {1, 2} has a support of 60% because it appears in 3 transactions out of 5 (it appears in t1, t2 and t5). A frequent itemset is an itemset that has a support no less than the maxsup parameter.

A perfectly rare itemset (aka sporadic itemset) is an itemset that is not a frequent itemset and that all its subsets are also not frequent itemsets. Moreover, it has to have a support higher or equal to the minsup threshold.

An association rule X==>Y is a relationship between two itemsets (sets of items) X and Y such that the intersection of X and Y is empty. The support of a rule is the number of transactions that contains X∪Y divided by the total number of transactions. The confidence of a rule is the number of transactions that contains X∪Y divided by the number of transactions that contain X.

A perfectly sporadic association rule X==>Y is an association rule such that the confidence is higher or equal to minconf and the support of any non empty subset of X∪Y is lower than maxsup.

For example, let's apply the algorithm with minsup = 0.1 %, maxsup of 60 % and minconf = 60 %.

The first step that the algorithm perform is to apply AprioriInverse algorithm with minsup = 0.1 % and maxsup of 60 %. The result is the following set of perfectly rare itemsets:

Perfectly Rare Itemsets Support
{3} 60 %
{4} 40 %
{5} 60 %
{4, 5} 40 %
{3, 5} 20 %

Then, the second step is to generate all perfectly sporadic association rules respecting minconf by using the perfectly rare itemsets found in the first step. The result is :

Rule Support Confidence
5 ==> 4 40 % 60 %
4 ==> 5 40 % 100 %

How to interpret the result?

For example, consider the rule 5 ==> 4. It means that if item 5 appears in a transaction, it is likely to be associated with item 4 with a confidence of 60 % (because 5 and 4 appears together in 40% of the transactions where 5 appears). Moreover, this rule has a support of 40 % because it appears in 40% of the transactions of this database.

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3
1 2 4 5

Consider the first line. It means that the first transaction is the itemset {1, 2, 4 and 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by an integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer. Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is the output file for this example:

5 ==> 4 #SUP: 2 #CONF: 0,6
4 ==> 5 #SUP: 2 #CONF: 1

For example, the first line indicates that the association rule {5} --> {4} has a support of 2 transactions and a confidence of 60 %. The second line follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Where can I get more information about this algorithm?

The AprioriInverse algorithm and how to generate sporadic rules are described in this paper:

Yun Sing Koh, Nathan Rountree: Finding Sporadic Rules Using Apriori-Inverse. PAKDD 2005: 97-106

Example 71 : Mining Closed Association Rules

How to run this example?

  • If you are using the graphical interface, (1) choose the "Closed_association_rules" algorithm , (2) select the input file "contextZart.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 60 %, minconf= 60% (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Closed_association_rules(using_fpclose) contextZart.txt output.txt 60% 60% in a folder containing spmf.jar and the example input file contextZart.txt.
  • To run this example with the source code version of SPMF, launch the file "MainTestClosedAssociationRulesWithFPClose_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

It is an algorithm for mining "closed association rules", which are a concise subset of all association rules.

What is the input of this algorithm?

The input is a transaction database (aka binary context) and two thresholds named minsup (a value in [0,1] that represents a percentage) and minconf (a value in [0,1] that represents a percentage).

A transaction database is a set of transactions. Each transaction is a set of distinct items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextZart.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3, 5}
t5 {1, 2, 3, 5}

What is the output of this algorithm?

Given the minimum support threshold (minsup) and minimum confidence threshold (minconf) set by the user, the algorithm returns the set of closed association rules that respect these thresholds. To explain what is a closed association rule, it is necessary to review some definitions.

An itemset is an unordered set of distinct items. The support of an itemset is the number of transactions that contain the itemset divided by the total number of transactions. For example, the itemset {1, 2} has a support of 60% because it appears in 3 transactions out of 5 (it appears in t1, t2 and t5). A closed itemset is an itemset that is strictly included in no itemset having the same support.

An association rule X==>Y is a relationship between two itemsets (sets of items) X and Y such that the intersection of X and Y is empty. The support of a rule X==>Y is the number of transactions that contains X∪Y divided by the total number of transactions. The confidence of a rule X==>Y is the number of transactions that contains X∪Y divided by the number of transactions that contain X. A closed association rule is an association rule of the form X ==> Y such that the union of X and Y is a closed itemset.

The algorithm returns all closed association rules such that their support and confidence are respectively higher or equal to the minsup and minconf thresholds set by the user.

For instance, by applying this algorithm with minsup = 60 %, minconf= 60%, we obtains 16 closed associations rules:

1 ==> 3 #SUP: 3 #CONF: 0.75 // which means that this rule has a support of 3 transactions and a confidence of 75 %
3 ==> 1 #SUP: 3 #CONF: 0.75 // which means that this rule has a support of 3 transactions and a confidence of 75 %
2 ==> 5 #SUP: 4 #CONF: 1.0 // which means that this rule has a support of 4 transactions and a confidence of 100 %
5 ==> 2 #SUP: 4 #CONF: 1.0 // ...
2 5 ==> 1 #SUP: 3 #CONF: 0.75
1 5 ==> 2 #SUP: 3 #CONF: 1.0
1 2 ==> 5 #SUP: 3 #CONF: 1.0
1 ==> 2 5 #SUP: 3 #CONF: 0.75
2 ==> 1 5 #SUP: 3 #CONF: 0.75
5 ==> 1 2 #SUP: 3 #CONF: 0.75
3 5 ==> 2 #SUP: 3 #CONF: 1.0
2 3 ==> 5 #SUP: 3 #CONF: 1.0
2 5 ==> 3 #SUP: 3 #CONF: 0.75
5 ==> 2 3 #SUP: 3 #CONF: 0.75
3 ==> 2 5 #SUP: 3 #CONF: 0.75
2 ==> 3 5 #SUP: 3 #CONF: 0.75

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3 5
1 2 3 5

This file contains five lines (five transactions). Consider the first line. It means that the first transaction is the itemset {1, 2, 4, 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by an integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer. Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is the output file for this example:

1 ==> 3 #SUP: 3 #CONF: 0.75
3 ==> 1 #SUP: 3 #CONF: 0.75
2 ==> 5 #SUP: 4 #CONF: 1.0
5 ==> 2 #SUP: 4 #CONF: 1.0
1 2 ==> 5 #SUP: 3 #CONF: 1.0
2 5 ==> 1 #SUP: 3 #CONF: 0.75
1 5 ==> 2 #SUP: 3 #CONF: 1.0
5 ==> 1 2 #SUP: 3 #CONF: 0.75
2 ==> 1 5 #SUP: 3 #CONF: 0.75
1 ==> 2 5 #SUP: 3 #CONF: 0.75
2 5 ==> 3 #SUP: 3 #CONF: 0.75
2 3 ==> 5 #SUP: 3 #CONF: 1.0
3 5 ==> 2 #SUP: 3 #CONF: 1.0
5 ==> 2 3 #SUP: 3 #CONF: 0.75
2 ==> 3 5 #SUP: 3 #CONF: 0.75
3 ==> 2 5 #SUP: 3 #CONF: 0.75

For example, the last line indicates that the association rule {3} --> {2, 5} has a support of 3 transactions and a confidence of 75 %. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Implementation details and performance

There are two versions of this algorithms implemented in SPMF. The first one uses CHARM for finding the frequent closed itemsets before generating the rules. The second one uses FPClose for finding the frequent closed itemsets before generating the rules. The version based on FPClose is generally faster than the version based on CHARM.

In the release version of SPMF, the algorithm "Closed_association_rules(using_fpclose)" denotes the version using FPClose, while "Closed_association_rules" denotes the version based on CHARM.

In the source code version of SPMF, the files "MainTestClosedAssociationRulesWithFPClose_saveToMemory" and "MainTestClosedAssociationRulesWithFPClose_saveToFile" denotes respectively the version using FPClose which saves the result to memory or to a file. Moreover, the files "MainTestClosedAssociationRules_saveToMemory" and "MainTestClosedAssociationRules_saveToFile" denotes respectively the version using CHARM which saves the result to memory or to a file.

Where can I get more information about closed association rules?

The following Ph.D. thesis proposed "closed association rules".

Szathmary, L. (2006). Symbolic Data Mining Methods with the Coron Platform. Szathmary, L. PhD thesis, University Henri Poincaré — Nancy 1, France.

Example 72 : Mining Minimal Non Redundant Association Rules

How to run this example?

  • If you are using the graphical interface, (1) choose the "MNR" algorithm , (2) select the input file "contextZart.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 60 %, minconf= 60% (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run MNR contextZart.txt output.txt 60% 60% in a folder containing spmf.jar and the example input file contextZart.txt.
  • To run this example with the source code version of SPMF, launch the file "MainTestMNRRules_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

This algorithm discover the set of "minimal non redundant association rules" (Kryszkiewicz, 1998), which is a lossless and compact set of association rules.

In this implementation we use the Zart algorithm for discovering closed itemsets and their associated generators. Then, this information is used to generate the "minimal non redundant association rules".

What is the input?

The input is a transaction database (aka binary context), a threshold named minconf (a value in [0,1] that represents a percentage) and a threshold named minsup (a value in [0,1] that represents a percentage).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextZart.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {1, 3}
t3 {1, 2, 3, 5}
t4 {2, 3, 5}
t5 {1, 2, 3, 5}

What is the output?

This algorithm returns the set of minimal non redundant association rules.

To explain what is the set of minimal non redundant association rules, it is necessary to review some definitions. An itemset is a set of distinct items. The support of an itemset is the number of times that it appears in the database divided by the total number of transactions in the database. For example, the itemset {1 3} has a support of 33 % because it appears in 2 out of 6 transactions from the database.

An association rule X--> Y is an association between two itemsets X and Y that are disjoint. The support of an association rule is the number of transactions that contains X and Y divided by the total number of transactions. The confidence of an association rule is the number of transactions that contains X and Y divided by the number of transactions that contains X.

A closed itemset is an itemset that is strictly included in no itemset having the same support. An itemset Y is the closure of an itemset X if Y is a closed itemset, X is a subset of Y and X and Y have the same support. A generator Y of a closed itemset X is an itemset such that (1) it has the same support as X and (2) it does not have a subset having the same support.

The set of minimal non redundant association rules is defined as the set of association rules of the form P1 ==> P2 / P1, where P1 is a generator of P2, P2 is a closed itemset, and the rule has a support and confidence respectively no less than minsup and minconf.

For example, by applying this algorithm with minsup = 60 %, minconf= 60% on the previous database, we obtains 14 minimal non redundant associations rules:

2 3 ==> 5 support:: 0.6 confidence: 1
3 5 ==> 2 support: 0.6 confidence: 1
1 ==> 3 support: 0.6 confidence: 0,75
1 ==> 2 5 support: 0.6 confidence: 0,75
1 2 ==> 5 support: 0.6 confidence: 1
1 5 ==> 2 support: 0.6 confidence: 1
3 ==> 1 support: 0.6 confidence: 0,75
3 ==> 2 5 support: 0.6 confidence: 0,75
2 ==> 3 5 support: 0.6 confidence: 0,75
2 ==> 1 5 support: 0.6 confidence: 0,75
2 ==> 5 support: 0.8 confidence: 1
5 ==> 2 3 support: 0.6 confidence: 0,75
5 ==> 1 2 support: 0.6 confidence: 0,75
5 ==> 2 support: 0.8 confidence: 1

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
1 3
1 2 3 5
2 3 5
1 2 3 5

This file contains five lines (five transactions). Consider the first line. It means that the first transaction is the itemset {1, 2, 4, 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by an integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an double value indicating a p. Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is the output file for this example:

2 ==> 5 #SUP: 0,8 #CONF: 1
2 ==> 3 5 #SUP: 0,6 #CONF: 0,75
2 ==> 1 5 #SUP: 0,6 #CONF: 0,75
5 ==> 2 #SUP: 0,8 #CONF: 1
5 ==> 2 3 #SUP: 0,6 #CONF: 0,75
5 ==> 1 2 #SUP: 0,6 #CONF: 0,75
3 ==> 2 5 #SUP: 0,6 #CONF: 0,75
3 ==> 1 #SUP: 0,6 #CONF: 0,75
2 3 ==> 5 #SUP: 0,6 #CONF: 1
3 5 ==> 2 #SUP: 0,6 #CONF: 1
1 2 ==> 5 #SUP: 0,6 #CONF: 1
1 5 ==> 2 #SUP: 0,6 #CONF: 1
1 ==> 3 #SUP: 0,6 #CONF: 0,75
1 ==> 2 5 #SUP: 0,6 #CONF: 0,75

For example, the last line indicates that the association rule {1} --> {2, 5} has a support of 60 % and a confidence of 75%. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Where can I get more information about closed association rules?

The following article provides detailed information about Minimal Non Redundant Association Rules:

M. Kryszkiewicz (1998). Representative Association Rules and Minimum Condition Maximum Consequence Association Rules. Proc. of PKDD '98, Nantes, France, September 23-26.

Example 73 : Mining Indirect Association Rules with the INDIRECT algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "Indirect_association_rules" algorithm, (2) select the input file "contextIndirect.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 60 %, ts = 50 % and minconf= 10% (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Indirect_association_rules contextIndirect.txt output.txt 60% 50% 10% in a folder containing spmf.jar and the example input file contextIndirect.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestIndirectRules_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is the INDIRECT algorithm?

Indirect (Tan et al., KDD 2000; Tan, Steinbach & Kumar, 2006, p.469) is an algorithm for discovering indirect associations between items in transactions databases.

Why this algorithm is important? Because traditional association rule mining algorithms focus on direct associations between itemsets. This algorithm can discover indirect associations, which can be useful in domains such as biology. Indirect association rule mining has various applications such as stock market analysis and competitive product analysis (Tan et al., 2000).

What is the input?

The input of the indirect algorithm is a transaction database and three parameters named minsup (a value in [0,1] that represents a percentage), ts (a value in [0,1] that represents a percentage) and minconf (a value in [0,1] that represents a percentage).

A transaction database is a set of transactions. Each transaction is a set of distinct items. For example, consider the following transaction database. It contains 5 transactions (t1, t2, ..., t5) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 4 and 5. This database is provided as the file contextIndirect.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 4, 5}
t2 {2, 3, 4}
t3 {1, 2, 4, 5}
t4 {5}
t5 {1, 2, 4, 5}

The three numeric parameters of the indirect algorithm are:

  • minsup : called the "mediator minimum support".
  • ts : called the "itempair minimum support"
  • minconf : representing the minimal confidence required for indirect associations (note that in the original article it uses the IS measure instead of the confidence).

What is the output?

The result is all indirect associations respecting the parameters minsup, ts and minconf. An indirect association has the form {x,y} ==> M, where x and y are single items and M is an itemset called the "mediator".

An indirect association has to respect the following conditions:

  • The number of transactions containing all items of {x}∪ M divided by the total number of transaction must be higher or equal to minsup.
  • The number of transactions containing all items of {y}∪ M divided by the total number of transaction must be higher or equal to minsup.
  • The number of transactions containing {x,y} divided by the total number of transaction must be smaller than ts.
  • The confidence of {x}with respect to M and {y}with respect M must be higher or equal to minconf. The confidence of an itemset X with respect to another itemset Y is defined as the number of transactions that contains X and Y divided by the number of transactions that contain X.

For example, by applying the indirect algorithm with minsup = 60 %, ts = 50 % and minconf= 10%, we obtain 3 indirect association rules:

  1. {1, 2 | {4}}, which means that 1 and 2 are indirectly associated by the mediator {4 }.
  2. {1, 5 | {4}}, which means that 1 and 5 are indirectly associated by the mediator {4 }.
  3. {2, 5 | {4}}, which means that 1 and 5 are indirectly associated by the mediator {4 }.

To see additional details about each of these three indirect rules, run this example.

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 4
2 3 4
1 2 4 5
4 5
1 2 4 5

This file contains five lines (five transactions). Consider the first line. It means that the first transaction is the itemset {1, 4}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an indirect association rule. Each line starts by "(a=x b=y | mediator=M )" indicating that the line represents the rule {x,y} ==> M, where x, y and M are integers representing items. Then, the keyword "#sup(a,mediator)=" is followed by the support of {x}∪ M expressed as a number of transactions (an integer). Then, the keyword "#sup(b,mediator)=" is followed by the support of {y}∪ M expressed as a number of transactions (an integer). Then, the keyword "#conf(a,mediator)= " is followed by the confidence of a with respect to the mediator, expressed as a double value in the [0, 1] interval. Then, the keyword "#conf(b,mediator)= " appears followed by the confidence of b with respect to the mediator, expressed as a double value in the [0, 1] interval.

For example, the output file of this example is:

(a=1 b=2 | mediator=4 ) #sup(a,mediator)= 3 #sup(b,mediator)= 3 #conf(a,mediator)= 1.0 #conf(b,mediator)= 1.0
(a=1 b=5 | mediator=4 ) #sup(a,mediator)= 3 #sup(b,mediator)= 3 #conf(a,mediator)= 1.0 #conf(b,mediator)= 1.0
(a=2 b=5 | mediator=4 ) #sup(a,mediator)= 3 #sup(b,mediator)= 3 #conf(a,mediator)= 1.0 #conf(b,mediator)= 1.0

This file contains three lines (three indirect association rules). Consider the first line. It represents that items 1 and 2 are indirectly associated by the item 4 as mediator. Furthermore, it indicates that the support of {1, 4} is 3 transactions, the support of {2,4} is 3 transactions, the confidence of item 1 with respect to item 4 is 100 % and the confidence of item 2 with respect to item 4 is 100%. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Implementation details

The implementation attempts to be as faithful as possible to the original algorithm, except that the confidence is used instead of the IS measure.

Note that some algorithms claimed to be more efficient than Indirect such as HI-Mine but they have not been implemented in SPMF.

Where can I get more information about indirect association rules?

The concept of indirect associations was proposed by Tan (2000) in this conference paper:

Pang-Ning Tan, Vipin Kumar, Jaideep Srivastava: Indirect Association: Mining Higher Order Dependencies in Data. PKDD 2000: 632-637

Moreover, note that the book "Introduction do data mining" by Tan, Steinbach and Kumar provides an overview of indirect association rules that is easy to read.

Example 74 : Hiding Sensitive Association Rules with the FHSAR algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "FHSAR" algorithm, (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50 %, minconf= 60% and sar_file = "sar.txt"(5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run FHSAR contextIGB.txt output.txt 50% 60% sar.txt in a folder containing spmf.jar and the example input file contextIGB.txt.
  • To run this example with the source code version of SPMF, launch the file "MainTestFHSAR_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is FHSAR?

FHSAR is an algorithm for hiding sensitive association rules in a transaction database.

What are the applications? For example, consider a company that want to release a transaction database to the public. But it does not want to disclose some sensitive associations between items that appear in the database and that could give a competitive advantage to their competitor. The FHSAR algorithm can hide these associations by modifying the database.

What is the input?

The FHSAR algorithm is designed to hide sensitive association rules in a transaction database so that they will not be found for a given minsup and minconf threshold generally used by association rule mining algorithms. The input are: minsup (a value in [0,1] that represents a percentage), minconf (a value in [0,1] that represents a percentage), a transaction database and some sensitive association rules to be hidden.

A transaction database is a set of transactions. Each transaction is a set of distinct items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1, 2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

An association rule X==>Yis an association between two sets of items X and Y such that X and Y are disjoint. The support of an association rule X==>Yis the number of transactions that contains both X and Y divided by the total number of transactions. The confidence of an association rule X==>Y is the number of transactions that contains both X and Y divided by the number of transactions that contain X. For example, the rule {1 2} ==> {4 5} has a support of 50 % because it appears in 3 transactions out of 5. Furthermore, it has a confidence of 75 % because {1 2} appears in 4 transactions and {1, 2, 4, 5} appears in 3 transactions.

What is the output?

The output is a new transaction database such that the sensitive rules will not be found if an association rule mining algorithm is applied with minsup and minconf.

For example, we can apply FHSAR with the parameters minsup = 0.5 and minconf = 0.60 to hide the following association rules provided in the file "sar.txt":

  • 4 ==> 1
  • 1 2 ==> 4 5
  • 5 ==> 2

the result is a new transaction database where these rules are hidden for the given thresholds minsup and minconf:

Transaction id Items
t1 {4, 5}
t2 {3, 5}
t3 {4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

Note that the result of the algorithm is not always the same because I use the HashSet data structure to represent transactions internally and this data structure do not keep the order. Therefore, the items that are removed may not be the same if the algorithm is run twice.

Input file format

This algorithm takes two files as input.

The first file is a text file containing transactions (a transaction database) (e.g. contextIGB.txt). Each line represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

This file contains six lines (six transactions). Consider the first line. It means that the first transaction is the itemset {1, 2, 4, 5}. The following lines follow the same format.

The second file is a text file containing sensitive association rules to be hidden (e.g. sar.txt). Each line is an association rule. First, the rule antecedent is written. It is an itemset, where each item is represented by a positive integer, and each item is separated from the following item by a single space. Note that it is assumed that items within an itemset cannot appear more than once and are sorted according to a total order. Then the keyword " ==> " appears followed by the rule consequent. The consequent is an itemset where each item is represented by a positive integer, and each item is separated from the following item by a single space. For example, consider the file sar.txt.

4 ==> 1
1 2 ==> 4 5
5 ==> 2

This file contains three lines (three association rules). The second line indicates that the rule {1, 2} ==> {4, 5} should be hidden by the FHSAR algorithm.

Output file format

The output file format is defined as follows. It is a text file representing a transaction database. Each line represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, an output file generated by the FHSAR algorithm is:

4 5
3 5
4 5
1 2 3 5
1 2 3 4 5
2 3 4

In this example, the first line represents the transaction {4, 5}. Other lines follow the same format.

Where can I get more information about the FHSAR algorithm?

This algorithm was proposed in this paper:

C.-C.Weng, S.-T. Chen, H.-C. Lo: A Novel Algorithm for Completely Hiding Sensitive Association Rules. ISDA (3) 2008: 202-208

Example 75 : Mining the Top-K Association Rules

How to run this example?

  • If you are using the graphical interface, (1) choose the "TopKRules" algorithm, (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set k = 2 and minconf = 0.8 (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run TopKRules contextIGB.txt output.txt 2 80% in a folder containing spmf.jar and the example input file contextIGB.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestTopKRules.java" in the package ca.pfv.SPMF.tests.

What is TopKRules?

TopKRules is an algorithm for discovering the top-k association rules appearing in a transaction database.

Why is it useful to discover top-k association rules? Because other association rule mining algorithms requires to set a minimum support (minsup) parameter that is hard to set (usually users set it by trial and error, which is time consuming). TopKRules solves this problem by letting users directly indicate k, the number of rules to be discovered instead of using minsup.

What is the input of TopKRules ?

TopKRules takes three parameters as input:

  • a transaction database,
  • a parameter k representing the number of association rules to be discovered (a positive integer),
  • a parameter minconf representing the minimum confidence that the association rules should have (a value in [0,1] representing a percentage).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

What is the output of TopKRules ?

TopKRules outputs the top-k association rules.

To explain what are top-k association rules, it is necessary to review some definitions. An itemset is a set of distinct items. The support of an itemset is the number of times that it appears in the database divided by the total number of transactions in the database. For example, the itemset {1 3} has a support of 33 % because it appears in 2 out of 6 transactions from the database.

An association rule X--> Y is an association between two itemsets X and Y that are disjoint. The support of an association rule is the number of transactions that contains X and Y divided by the total number of transactions. The confidence of an association rule is the number of transactions that contains X and Y divided by the number of transactions that contains X.

The top-k association rules are the k most frequent association rules in the database having a confidence higher or equal to minconf.

For example, if we run TopKRules with k = 2 and minconf = 0.8, we obtain the top-2 rules in the database having a confidence higher or equals to 80 %.

  • 2 ==> 5, which have a support of 5 (it appears in 5 sequences) and a confidence of 83%
  • 5 ==> 2, which have a support of 5 (it appears in 5 sequences) and a confidence of 100 %

For instance, the rule 2 ==>5 means that if item 2 appears, it is likely to be associated with item 5 with a confidence of 83% in a transaction. Moreover, this rule has a support of 83 % because it appears in five transactions (S1, S2 and S3) out of the six transactions contained in this database.

It is important to note that for some values of k, the algorithm may return slightly more than k rules. This is can happen if several rules have exactly the same support.

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

Consider the first line. It means that the first transaction is the itemset {1, 2, 4 and 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by a positive integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer (a number of transactions). Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is a few lines from the output file if we run TopKRules on contextIGB.txt with k=2 and minconf=0.8 (80 %):

2 ==> 5 #SUP: 5 #CONF: 0.8333333333333334
5 ==> 2 #SUP: 5 #CONF: 1.0

For example, the first line indicates that the association rule {2} --> {5} has a support of 5 transactions and a confidence of 83.3 %. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

TopKRules is a very efficient algorithm for mining the top-k association rules.

It provides the benefits that it is very intuitive to use. It should be noted that the problem of top-k association rule mining is more computationally expensive than the problem of association rule mining. Using TopKRules is recommended for k values of up to 5000, depending on the datasets.

Besides, note that there is a variation of TopKRules named TNR that is available in SPMF. The improvement in TNR is that it eliminates some association rules that are deemed "redundant" (rules that are included in other rules having the same support and confidence - see the TNR example for the formal definition). Using TNR is more costly than using TopKRules but it brings the benefit of eliminating a type of redundancy in results.

Where can I get more information about this algorithm?

The TopKRules algorithm was proposed in this paper:

Fournier-Viger, P., Wu, C.-W., Tseng, V. S. (2012). Mining Top-K Association Rules. Proceedings of the 25th Canadian Conf. on Artificial Intelligence (AI 2012), Springer, LNAI 7310, pp. 61-73.

Example 76 : Mining the Top-K Non-Redundant Association Rules

How to run this example?

  • If you are using the graphical interface, (1) choose the "TNR" algorithm, (2) select the input file "contextIGB.txt", (3) set the output file name (e.g. "output.txt") (4) set k = 30,minconf = 0.5, and delta = 2 (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run TNR contextIGB.txt output.txt 30 0.5 2 in a folder containing spmf.jar and the example input file contextIGB.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestTNR.java" in the package ca.pfv.SPMF.tests.

What is TopKRules?

TNR is an algorithm for discovering the top-k non-redundant association rules appearing in a transaction database. It is an approximate algorithm in the sense that it always generates non-redundant rules. But these rules may not always be the top-k non-redundant association rules. TNR uses a parameter named delta, which is a positive integer >=0 that can be used to improve the chance that the result is exact (the higher the delta value, the more chances that the result will be exact).

Why is it important to discover top-k non-redundant association rules? Because other association rule mining algorithms requires that the user set a minimum support (minsup) parameter that is hard to set (usually users set it by trial and error, which is time consuming). Moreover, the result of association rule mining algorithms usually contains a high level of redundancy (for example, thousands of rules can be found that are variation of other rules having the same support and confidence). The TNR algorithm provide a solution to both of these problems by letting users directly indicate k, the number of rules to be discovered, and by eliminating redundancy in results.

What is the input of TNR ?

TNR takes four parameters as input:

  • a transaction database,
  • a parameter k representing the number of rules to be discovered (a positive integer >= 1),
  • a parameter minconf representing the minimum confidence that association rules should have (a value in [0,1] representing a percentage).
  • a parameter delta (a positive integer >=0) that is used to increase the chances of having an exact result (because the TNR algorithm is approximate).

A transaction database is a set of transactions. Each transaction is a set of items. For example, consider the following transaction database. It contains 6 transactions (t1, t2, ..., t5, t6) and 5 items (1,2, 3, 4, 5). For example, the first transaction represents the set of items 1, 2, 4 and 5. This database is provided as the file contextIGB.txt in the SPMF distribution. It is important to note that an item is not allowed to appear twice in the same transaction and that items are assumed to be sorted by lexicographical order in a transaction.

Transaction id Items
t1 {1, 2, 4, 5}
t2 {2, 3, 5}
t3 {1, 2, 4, 5}
t4 {1, 2, 3, 5}
t5 {1, 2, 3, 4, 5}
t6 {2, 3, 4}

What is the output of TNS ?

TNR outputs an approximation of the k most frequent non redundant association rules having a confidence higher or equal to minconf.

To explain what are top-k non redundant association rules, it is necessary to review some definitions. An itemset is a set of distinct items. The support of an itemset is the number of times that it appears in the database divided by the total number of transactions in the database. For example, the itemset {1 3} has a support of 33 % because it appears in 2 out of 6 transactions from the database.

An association rule X--> Y is an association between two itemsets X and Y that are disjoint. The support of an association rule is the number of transactions that contains X and Y divided by the total number of transactions. The confidence of an association rule is the number of transactions that contains X and Y divided by the number of transactions that contains X.

An association rule ra: X → Y is redundant with respect to another rule rb : X1 → Y1 if and only if:

  • conf(ra) = conf(rb)
  • sup(ra) = sup(rb)
  • X1 ⊆ X ∧ Y ⊆ Y1.

The top-k non redundant association rules are the k most non-redundant frequent association rules in the database having a confidence higher or equal to minconf.

For example, If we run TNR with k = 10 and minconf = 0.5 and delta = 2, the following set of rules is found

4, ==> 2,  sup= 4  conf= 1.0
2, ==> 1,5, sup= 4 conf=0.66
2, ==> 5, sup= 5 conf= 0.8333333333333334
5, ==> 2, sup= 5 conf= 1.0
5, ==> 1,2, sup= 4 conf= 0.8
1, ==> 2,5, sup= 4 conf= 1.0
2, ==> 3, sup= 4 conf=0.66
2, ==> 4, sup= 4 conf=0.66
3, ==> 2, sup= 4 conf= 1.0
1,4, ==> 2,5, sup= 3 conf= 1.0

For instance, the association rule 2 ==> 1 5 means that if items 2 appears, it is likely to be associated with item 1 and item 5 with a confidence of 66 %. Moreover, this rule has a support 66 % (sup = 4) because it appears in three transaction (S1, S2 and S3) out of six transactions contained in this database.

Note that for some values of k and some datasets, TNR may return more than k association rules. This can happen if several rules have exactly the same support, and it is normal. It is also possible that the algorithm returns slightly less than k association rules in some circumstances because the algorithm is approximate.

Input file format

The input file format is a text file containing transactions. Each lines represents a transaction. The items in the transaction are listed on the corresponding line. An item is represented by a positive integer. Each item is separated from the following item by a space. It is assumed that items are sorted according to a total order and that no item can appear twice in the same transaction. For example, for the previous example, the input file is defined as follows:

1 2 4 5
2 3 5
1 2 4 5
1 2 3 5
1 2 3 4 5
2 3 4

Consider the first line. It means that the first transaction is the itemset {1, 2, 4 and 5}. The following lines follow the same format.

Note that it is also possible to use the ARFF format as an alternative to the default input format. The specification of the ARFF format can be found here. Most features of the ARFF format are supported except that (1) the character "=" is forbidden and (2) escape characters are not considered. Note that when the ARFF format is used, the performance of the data mining algorithms will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file, where each line represents an association rule. On each line, the items of the rule antecedent are first listed. Each item is represented by a positive integer, followed by a single space. After, that the keyword "==>" appears followed by a space. Then, the items of the rule consequent are listed. Each item is represented by an integer, followed by a single space. Then, the keyword " #SUP: " appears followed by the support of the rule represented by an integer (a number of transactions). Then, the keyword " #CONF: " appears followed by the confidence of the rule represented by a double value (a value between 0 and 1, inclusively). For example, here is a few lines from the output file if we run TopKRules on contextIGB.txt with k=3 and minconf=0.8 (80 %):

2 ==> 4 #SUP: 4 #CONF:0.66
5 ==> 1 2 #SUP: 4 #CONF: 0.8
5 ==> 2 #SUP: 5 #CONF: 1.0
2 ==> 5 #SUP: 5 #CONF: 0.8333333333333334
2 ==> 1 5 #SUP: 4 #CONF:0.66
1 ==> 2 5 #SUP: 4 #CONF: 1.0
2 ==> 3 #SUP: 4 #CONF:0.66
3 ==> 2 #SUP: 4 #CONF: 1.0
4 ==> 2 #SUP: 4 #CONF: 1.0
4 5 ==> 1 2 #SUP: 3 #CONF: 1.0

For example, the first line indicates that the association rule {2} --> {4} has a support of 4 transactions and a confidence of 66.66 %. The other lines follow the same format.

Note that if the ARFF format is used as input instead of the default input format, the output format will be the same except that items will be represented by strings instead of integers.

Performance

TNR is an efficient algorithm. It is based on the TopKRules algorithm for discovering top-k association rule. The main difference between TNR and TopKRules is that TNR includes additional strategies to eliminate redundancy in results, and that TNR is an approximate algorithm, while TopKRules is not.

TNR and TopKRules are more intuitive to use than regular association rule mining algorithms. However, it should be noted that the problem of top-k association rule mining is more computationally expensive than the problem of association rule mining. Therefore, it is recommended to use TNR or TopKRules for k values of up to 5000 depending on the dataset. If more rules should be found, it could be better to find association rules with a classical association rule mining algorithm like FPGrowth, for more efficiency.

Where can I get more information about this algorithm?

The TNR algorithm is described in this paper:

Fournier-Viger, P., Tseng, V.S. (2012). Mining Top-K Non-Redundant Association Rules. Proc. 20th International Symposium on Methodologies for Intelligent Systems (ISMIS 2012), Springer, LNCS 7661, pp. 31- 40.

Example 77 : Clustering Values with the K-Means algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "KMeans" algorithm, (2) select the input file "inputDBScan2.txt", (3) set the output file name (e.g. "output.txt") (4) set k =3 and distance function = euclidian, and (5) click "Run algorithm".
    Note that there is also an optional parameter called "separator". It can be used to read other types of input files where values are not separated by spaces. For example, for a file separated by the ',' character, the parameter "separator" would have to be set to ",".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run KMeans inputDBScan2.txt output.txt 3 euclidian in a folder containing spmf.jar and the example input file inputDBScan2.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestKMeans_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is K-Means?

K-Means is one of the most famous clustering algorithm. It is used to automatically separate a set of instances (vectors of double values) into groups of instances (clusters) according to their similarity. Thus, K-Means is used to automatically group similar instances together into clusters.

In this implementation the user can choose between various distance functions to assess the similarity between vectors. SPMF offers the Euclidian distance, correlation distance, cosine distance, Manathan distance and Jaccard distance.

What is the input?

K-Means takes as input a set of instances having a name and containing one or more double values, a parameter K (a positive integer >=1) indicating the number of clusters to be created, and a distance function.

The input file format of K-Means is a text file containing several instances.

The first lines (optional) specify the name of the attributes used for describing the instances. In this example, two attributes will be used, named X and Y. But note that more than two attributes could be used. Each attribute is specified on a separated line by the keyword "@ATTRIBUTEDEF=", followed by the attribute name

Then, each instance is described by two lines. The first line (which is optional) contains the string "@NAME=" followed by the name of the instance. Then, the second line provides a list of double values separated by single spaces.

An example of input is provided in the file "inputDBScan2.txt" of the SPMF distribution. It contains 31 instances, each described by two attribute called X and Y.

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
@NAME=Instance1
1 1
@NAME=Instance2
0 1
@NAME=Instance3
1 0
@NAME=Instance4
11 12
@NAME=Instance5
11 13
@NAME=Instance6
13 13
@NAME=Instance7
12 8.5
@NAME=Instance8
13 8
@NAME=Instance9
13 9
@NAME=Instance10
13 7
@NAME=Instance11
11 7
@NAME=Instance12
8 2
@NAME=Instance13
9 2
@NAME=Instance14
10 1
@NAME=Instance15
7 13
@NAME=Instance16
5 9
@NAME=Instance17
16 16
@NAME=Instance18
11.5 8
@NAME=Instance20
13 10
@NAME=Instance21
12 13
@NAME=Instance21
14 12.5
@NAME=Instance22
14.5 11.5
@NAME=Instance23
15 10.5
@NAME=Instance24
15 9.5
@NAME=Instance25
12 9.5
@NAME=Instance26
10.5 11
@NAME=Instance27
10 10.5
@NAME=Instance28
9 3
@NAME=Instance29
9 4
@NAME=Instance30
9 5

For example, the first instance is named "Instance1", and it is a vector with two values: 1 and 1 for the attributes X and Y, respectively.

This input file represents a set of 2D points. But note that, it is possible to use more than two attributes to describe instances. To give a better idea of what the input file looks like, here is a visual representation:

The K-Means algorithm will group the instances according to their similarity. To do this, it is also necessary to specify the distance function to be used for comparing the instances. The distance function can be the Euclidian distance, correlation distance, cosine distance, Manathan distance and Jaccard distance. In the command line or GUI of SPMF, the distance function is specified by using one of these keywords: "euclidian", "correlation", "cosine", "manathan" and "jaccard" as parameter. In this example, the euclidian distance is used.

What is the output?

K-Means groups instances in clusters according to their similarity. In SPMF, the similarity is defined according to the distance function chosen by the user such as the Euclidian distance. K-Means returns K clusters or less.

Note that running K-Means with the same data does not always generate the same result because K-Means initializes clusters randomly.

By running K-Means on the previous input file and K=3, we can obtain the following output file containing 3 clusters:

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
[Instance12 8.0 2.0][Instance13 9.0 2.0][Instance14 10.0 1.0][Instance28 9.0 3.0][Instance29 9.0 4.0][Instance30 9.0 5.0]
[Instance4 11.0 12.0][Instance5 11.0 13.0][Instance6 13.0 13.0][Instance7 12.0 8.5][Instance8 13.0 8.0][Instance9 13.0 9.0][Instance11 11.0 7.0][Instance15 7.0 13.0][Instance16 5.0 9.0][Instance17 16.0 16.0][Instance18 11.5 8.0][Instance20 13.0 10.0][Instance21 12.0 13.0][Instance21 14.0 12.5][Instance22 14.5 11.5][Instance23 15.0 10.5][Instance24 15.0 9.5][Instance25 12.0 9.5][Instance26 10.5 11.0][Instance27 10.0 10.5][Instance10 13.0 7.0]
[Instance1 1.0 1.0][Instance2 0.0 1.0][Instance3 1.0 0.0]

The output file format is defined as follows. The first few lines indicate the attribute names. Each attribute is specified on a separated line with the keyword "ATTRIBUTEDEF=" followed by the attribute name (a string). Then, the list of clusters is indicated. Each cluster is specified on a separated line, listing the instances contained in the cluster. An instance is a name followed by a list of double values separated by " " and between the "[" and "]" characters.

The clusters found by the algorithm can be viewed visually using the "Cluster Viewer" provided in SPMF. If you are using the graphical interface of SPMF, click the checkbox "Cluster Viewer" before pressing the "Run Algorithm" button. The result will be displayed in the Cluster Viewer.

As it can be seen in this example, the result somewhat make sense, as points close to each other are in the same cluster.

Applying K-Means to time series

Note that the K-Means algorithm implementation in SPMF can also be applied to time series database such as the file contextSAX.txt in the SPMF distribution. To apply K-Means to time series, it is necessary to set the "separator" parameter of the K-Means algorithm to "," since time series files separate values by "," instead of spaces.

Where can I get more information about K-Means?

K-Means was proposed by MacQueen in 1967. K-means is one of the most famous data mining algorithm. It is described in almost all data mining books that focus on algorithms, and on many websites. By searching on the web, you will find plenty of resources explaining K-Means.

Example 78 : Clustering Values with the Bisecting K-Means algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "BisectingKMeans" algorithm, (2) select the input file "inputDBScan2.txt", (3) set the output file name (e.g. "output.txt") (4) set k =3, distance function = euclidian, iter = 10 and(5) click "Run algorithm".
    Note that there is also an optional parameter called "separator". It can be used to read other types of input files where values are not separated by spaces. For example, for a file separated by the ',' character, the parameter "separator" would have to be set to ",".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run BisectingKMeans inputDBScan2.txt output.txt 3 euclidian 10 in a folder containing spmf.jar and the example input file inputDBScan2.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestBisectingKMeans_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is Bisecting K-Means?

K-Means is one of the most famous clustering algorithm. It is used to separate a set of instances (vectors of double values) into groups of instances (clusters) according to their similarity.

The Bisecting K-Means algorithm is a variation of the regular K-Means algorithm that is reported to perform better for some applications. It consists of the following steps: (1) pick a cluster, (2) find 2-subclusters using the basic K-Means algorithm, * (bisecting step), (3) repeat step 2, the bisecting step, for ITER times and take the split that produces the clustering, (4) repeat steps 1,2,3 until the desired number of clusters is reached.

In this implementation the user can choose between various distance functions to assess the distance between vectors. SPMF offers the Euclidian distance, correlation distance, cosine distance, Manathan distance and Jaccard distance.

What is the input?

Bisecting K-Means takes as input a set of instances (each having a name and containing one or more double values), a parameter K (a positive integer >=1) indicating the number of clusters to be created, a distance function, and the parameter ITER.

The input file format of K-Means is a text file containing several instances.

The first lines (optional) specify the name of the attributes used for describing the instances. In this example, two attributes will be used, named X and Y. But note that more than two attributes could be used. Each attribute is specified on a separated line by the keyword "@ATTRIBUTEDEF=", followed by the attribute name

Then, each instance is described by two lines. The first line (which is optional) contains the string "@NAME=" followed by the name of the instance. Then, the second line provides a list of double values separated by single spaces.

An example of input is provided in the file "inputDBScan2.txt" of the SPMF distribution. It contains 31 instances, each described by two attribute called X and Y.

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
@NAME=Instance1
1 1
@NAME=Instance2
0 1
@NAME=Instance3
1 0
@NAME=Instance4
11 12
@NAME=Instance5
11 13
@NAME=Instance6
13 13
@NAME=Instance7
12 8.5
@NAME=Instance8
13 8
@NAME=Instance9
13 9
@NAME=Instance10
13 7
@NAME=Instance11
11 7
@NAME=Instance12
8 2
@NAME=Instance13
9 2
@NAME=Instance14
10 1
@NAME=Instance15
7 13
@NAME=Instance16
5 9
@NAME=Instance17
16 16
@NAME=Instance18
11.5 8
@NAME=Instance20
13 10
@NAME=Instance21
12 13
@NAME=Instance21
14 12.5
@NAME=Instance22
14.5 11.5
@NAME=Instance23
15 10.5
@NAME=Instance24
15 9.5
@NAME=Instance25
12 9.5
@NAME=Instance26
10.5 11
@NAME=Instance27
10 10.5
@NAME=Instance28
9 3
@NAME=Instance29
9 4
@NAME=Instance30
9 5

For example, the first instance is named "Instance1", and it is a vector with two values: 1 and 1 for the attributes X and Y, respectively.

This input file represents a set of 2D points. But note that, it is possible to use more than two attributes to describe instances. To give a better idea of what the input file looks like, here is a visual representation:

The Bisecting K-Means algorithm will group the instances according to their similarity. To do this, it is also necessary to specify the distance function to be used for comparing the instances. The distance function can be the Euclidian distance, correlation distance, cosine distance, Manathan distance and Jaccard distance. In the command line or GUI of SPMF, the distance function is specified by using one of these keywords: "euclidian", "correlation", "cosine", "manathan" and "jaccard" as parameter. In this example, the euclidian distance is used.

The ITER specifies how much times the algorithm should repeat a split to keep the best split. If it is set to a high value it should provide better results but it should be more slow. Splits are evaluated using the Squared Sum of Errors (SSE).

What is the output?

Bisecting K-Means groups vectors in clusters according to their similarity. In SPMF, the similarity is defined according to the distance function chosen by the user such as the Euclidian distance. K-Means returns K clusters or less.

Note that running Bisecting K-Means with the same data does not always generate the same result because Bisecting K-Means initializes clusters randomly.

By running Bisecting K-Means on the previous input file, we can obtain the following output file containing 3 clusters:

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
[Instance1 1.0 1.0][Instance2 0.0 1.0][Instance3 1.0 0.0][Instance12 8.0 2.0][Instance13 9.0 2.0][Instance14 10.0 1.0][Instance16 5.0 9.0][Instance28 9.0 3.0][Instance29 9.0 4.0][Instance30 9.0 5.0]
[Instance9 13.0 9.0][Instance24 15.0 9.5][Instance7 12.0 8.5][Instance8 13.0 8.0][Instance10 13.0 7.0][Instance11 11.0 7.0][Instance18 11.5 8.0][Instance20 13.0 10.0][Instance25 12.0 9.5][Instance23 15.0 10.5]
[Instance4 11.0 12.0][Instance5 11.0 13.0][Instance6 13.0 13.0][Instance17 16.0 16.0][Instance21 12.0 13.0][Instance21 14.0 12.5][Instance22 14.5 11.5][Instance15 7.0 13.0][Instance26 10.5 11.0][Instance27 10.0 10.5]

The output file format is defined as follows. The first few lines indicate the attribute names. Each attribute is specified on a separated line with the keyword "ATTRIBUTEDEF=" followed by the attribute name (a string). Then, the list of clusters is indicated. Each cluster is specified on a separated line, listing the instances contained in the cluster. An instance is a name followed by a list of double values separated by " " and between the "[" and "]" characters.

The clusters found by the algorithm can be viewed visually using the "Cluster Viewer" provided in SPMF. If you are using the graphical interface of SPMF, click the checkbox "Cluster Viewer" before pressing the "Run Algorithm" button. The result will be displayed in the Cluster Viewer.

As it can be seen in this example, the result somewhat make sense, as points close to each other are in the same cluster.

Applying Bisecting K-Means to time series

Note that the Bisecting K-Means algorithm implementation in SPMF can also be applied to time series database such as the file contextSAX.txt in the SPMF distribution. To apply this algorithm to time series, it is necessary to set the "separator" parameter of this algorithm to "," since time series files separate values by "," instead of separating by spaces.

Where can I get more information about Bisecting K-Means ?

The original K-Means was proposed by MacQueen in 1967. K-means is one of the most famous data mining algorithm. It is described in almost all data mining books that focus on algorithms, and on many websites. By searching on the web, you will find plenty of resources explaining K-Means.

The Bisecting K-Means algorithms is described in this paper:

A comparison of document clustering techniques", M. Steinbach, G. Karypis and V. Kumar. Workshop on Text Mining, KDD, 2000.

Example 79 : Clustering Values with the DBScan algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "DBScan" algorithm, (2) select the input file "inputDBScan2.txt", (3) set the output file name (e.g. "output.txt") (4) set minPts =2 and epsilon = 2, and(5) click "Run algorithm".
    Note that there is also an optional parameter called "separator". It can be used to read other types of input files where values are not separated by spaces. For example, for a file separated by the ',' character, the parameter "separator" would have to be set to ",".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run DBScan inputDBScan2.txt output.txt 2 2 in a folder containing spmf.jar and the example input file inputDBScan2.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestDBScan_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is DBScan?

DBScan is an old but famous clustering algorithm. It is used to find clusters of points based on the density.

Implementation note: To avoid having a O(n^2) time complexity, this implementation uses a KD-Tree to store points internally.

What is the input?

DBScan takes as input (1) a set of instances having a name and containing one or more double values, (2) a parameter minPts (a positive integer >=1) indicating the number of points that a core point need to have in its neighborhood (see paper about DBScan for more details) and (3) a radius epsilon that define the neighborhood of a point.

The input file format is is a text file containing several instances.

The first lines (optional) specify the name of the attributes used for describing the instances. In this example, two attributes will be used, named X and Y. But note that more than two attributes could be used. Each attribute is specified on a separated line by the keyword "@ATTRIBUTEDEF=", followed by the attribute name

Then, each instance is described by two lines. The first line (which is optional) contains the string "@NAME=" followed by the name of the instance. Then, the second line provides a list of double values separated by single spaces.

An example of input is provided in the file "inputDBScan2.txt" of the SPMF distribution. It contains 31 instances, each described by two attribute called X and Y.

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
@NAME=Instance1
1 1
@NAME=Instance2
0 1
@NAME=Instance3
1 0
@NAME=Instance4
11 12
@NAME=Instance5
11 13
@NAME=Instance6
13 13
@NAME=Instance7
12 8.5
@NAME=Instance8
13 8
@NAME=Instance9
13 9
@NAME=Instance10
13 7
@NAME=Instance11
11 7
@NAME=Instance12
8 2
@NAME=Instance13
9 2
@NAME=Instance14
10 1
@NAME=Instance15
7 13
@NAME=Instance16
5 9
@NAME=Instance17
16 16
@NAME=Instance18
11.5 8
@NAME=Instance20
13 10
@NAME=Instance21
12 13
@NAME=Instance21
14 12.5
@NAME=Instance22
14.5 11.5
@NAME=Instance23
15 10.5
@NAME=Instance24
15 9.5
@NAME=Instance25
12 9.5
@NAME=Instance26
10.5 11
@NAME=Instance27
10 10.5
@NAME=Instance28
9 3
@NAME=Instance29
9 4
@NAME=Instance30
9 5

For example, the first instance is named "Instance1", and it is a vector with two values: 1 and 1 for the attributes X and Y, respectively.

This input file represents a set of 2D points. But note that, it is possible to use more than two attributes to describe instances. To give a better idea of what the input file looks like, here is a visual representation:

The distance function used by Optics is the Euclidian distance

What is the output?

DBScans groups vectors (points) in clusters based on density and distance between points.

Note that it is normal that DBScan may generate a cluster having less than minPts (this happens if the neibhoors of a core points get "stolen" by another cluster).

Note also that DBScan eliminate points that are seen as noise (a point having less than minPts neighboors within a radius of epsilon)

By running DBScan on the previous input file and minPts =2 and epsilon=2, we obtain the following output file containing 4 clusters:

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
[Instance1 1.0 1.0][Instance3 1.0 0.0][Instance2 0.0 1.0]
[Instance14 10.0 1.0][Instance13 9.0 2.0][Instance28 9.0 3.0][Instance29 9.0 4.0][Instance12 8.0 2.0][Instance30 9.0 5.0]
[Instance27 10.0 10.5][Instance26 10.5 11.0][Instance4 11.0 12.0][Instance5 11.0 13.0][Instance21 12.0 13.0][Instance6 13.0 13.0][Instance21 14.0 12.5][Instance22 14.5 11.5][Instance23 15.0 10.5][Instance24 15.0 9.5]
[Instance11 11.0 7.0][Instance18 11.5 8.0][Instance7 12.0 8.5][Instance25 12.0 9.5][Instance8 13.0 8.0][Instance9 13.0 9.0][Instance10 13.0 7.0][Instance20 13.0 10.0]

The output file format is defined as follows. The first few lines indicate the attribute names. Each attribute is specified on a separated line with the keyword "ATTRIBUTEDEF=" followed by the attribute name (a string). Then, the list of clusters is indicated. Each cluster is specified on a separated line, listing the instances contained in the cluster. An instance is a name followed by a list of double values separated by " " and between the "[" and "]" characters.

The clusters found by the algorithm can be viewed visually using the "Cluster Viewer" provided in SPMF. If you are using the graphical interface of SPMF, click the checkbox "Cluster Viewer" before pressing the "Run Algorithm" button. The result will be displayed in the Cluster Viewer.

As it can be seen in this example, the result somewhat make sense. Points that are close to each other have put in the same clusters. An interesting thing about DBScan is that it can find clusters of various shapes.

Applying DBSCAN to time series

Note that the DBScan algorithm implementation in SPMF can also be applied to time series database such as the file contextSAX.txt in the SPMF distribution. To apply this algorithm to time series, it is necessary to set the "separator" parameter of this algorithm to "," since time series files separate values by "," instead of separating by spaces.

Where can I get more information about DBScan?

DBScan is a most famous data mining algorithm for clustering. It is described in almost all data mining books that focus on algorithms, and on many websites. The original article describing DBScan is:

Ester, Martin; Kriegel, Hans-Peter; Sander, Jörg; Xu, Xiaowei (1996). Simoudis, Evangelos; * Han, Jiawei; Fayyad, Usama M., eds. A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96). AAAI Press. pp. 226–231.

Example 80 : Using Optics to extract a cluster-ordering of points and DB-Scan style clusters

What is OPTICS?

OPTICS is a classic clustering algorithm. It takes as input a set of instances (vectors of double values) and output a cluster-ordering of instances (points), that is a total order on the set of instances.

This "cluster-ordering" of points can then be used to generate density-based clusters similar to those generated by DBScan.

In the paper describing Optics, the authors also proposed authors tasks that can be done using the cluster-ordering of points such as interactive visualization and automatically extracting hierarchical clusters. Those tasks are not implemented here.

Implementation note: To avoid having a O(n^2) time complexity, this implementation uses a KD-Tree to store points internally.

In this implementation the user can choose between various distance functions to assess the similarity between vectors. SPMF offers the Euclidian distance, correlation distance, cosine distance, Manathan distance and Jaccard distance.

How to run this example?

To generate a cluster-ordering of points using OPTICS:

  • If you are using the graphical interface, (1) choose the "OPTICS-cluster-ordering" algorithm, (2) select the input file "inputDBScan2.txt", (3) set the output file name (e.g. "output.txt") (4) set minPts =2 and epsilon = 2, and (5) click "Run algorithm".
    Note that there is also an optional parameter called "separator". It can be used to read other types of input files where values are not separated by spaces. For example, for a file separated by the ',' character, the parameter "separator" would have to be set to ",".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run OPTICS-cluster-ordering inputDBScan2.txt output.txt 2 2 in a folder containing spmf.jar and the example input file inputDBScan2.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestOPTICS_extractClusterOrdering_saveToFile.java" in the package ca.pfv.SPMF.tests.

To generate a DB-Scan style cluster of points using OPTICS:

  • If you are using the graphical interface, (1) choose the "OPTICS-dbscan-clusters" algorithm, (2) select the input file "inputDBScan2.txt", (3) set the output file name (e.g. "output.txt") (4) set minPts =2, epsilon = 2 and epsilonPrime = 5, and(5) click "Run algorithm".
    Note that there is also an optional parameter called "separator". It can be used to read other types of input files where values are not separated by spaces. For example, for a file separated by the ',' character, the parameter "separator" would have to be set to ",".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run OPTICS-dbscan-clusters inputDBScan2.txt output.txt 2 2 5 in a folder containing spmf.jar and the example input file inputDBScan2.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestOPTICS_extractDBScan_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is the input?

Optics takes as input (1) a set of instances (points) having a name and containing one or more double values, (2) a parameter minPts (a positive integer >=1) indicating the number of instances (points) that a core point need to have in its neighborhood (see paper about Optics for more details) and (3) a radius epsilon that define the neighborhood of a point. If clusters are generated, an extra parameter named epsilonPrime is also taken as parameter. This latter parameter can be set to the same value as epsilon or a different value (see paper for details).

The input file format is a text file containing several instances.

The first lines (optional) specify the name of the attributes used for describing the instances. In this example, two attributes will be used, named X and Y. But note that more than two attributes could be used. Each attribute is specified on a separated line by the keyword "@ATTRIBUTEDEF=", followed by the attribute name

Then, each instance is described by two lines. The first line (which is optional) contains the string "@NAME=" followed by the name of the instance. Then, the second line provides a list of double values separated by single spaces.

An example of input is provided in the file "inputDBScan2.txt" of the SPMF distribution. It contains 31 instances, each described by two attribute called X and Y.

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
@NAME=Instance1
1 1
@NAME=Instance2
0 1
@NAME=Instance3
1 0
@NAME=Instance4
11 12
@NAME=Instance5
11 13
@NAME=Instance6
13 13
@NAME=Instance7
12 8.5
@NAME=Instance8
13 8
@NAME=Instance9
13 9
@NAME=Instance10
13 7
@NAME=Instance11
11 7
@NAME=Instance12
8 2
@NAME=Instance13
9 2
@NAME=Instance14
10 1
@NAME=Instance15
7 13
@NAME=Instance16
5 9
@NAME=Instance17
16 16
@NAME=Instance18
11.5 8
@NAME=Instance20
13 10
@NAME=Instance21
12 13
@NAME=Instance21
14 12.5
@NAME=Instance22
14.5 11.5
@NAME=Instance23
15 10.5
@NAME=Instance24
15 9.5
@NAME=Instance25
12 9.5
@NAME=Instance26
10.5 11
@NAME=Instance27
10 10.5
@NAME=Instance28
9 3
@NAME=Instance29
9 4
@NAME=Instance30
9 5

For example, the first instance is named "Instance1", and it is a vector with two values: 1 and 1 for the attributes X and Y, respectively.

This input file represents a set of 2D points. But note that, it is possible to use more than two attributes to describe instances. To give a better idea of what the input file looks like, here is a visual representation:

The distance function used by Optics is the Euclidian distance

What is the output?

Optics generates a a so-called cluster-ordering of points, which is a list of points with their reachability distances. For example, for minPts = 2 and epsilon = 2, the following cluster ordering is generated, where each line indicates respresent an instance (a name and its vector of values) and a reachability distance. Note that a reachability distance equals to "infinity" means "UNDEFINED" in the original paper.

Cluster orderings
Instance2 0.0 1.0 Infinity
Instance1 1.0 1.0 1.0
Instance3 1.0 0.0 1.0
Instance14 10.0 1.0 Infinity
Instance13 9.0 2.0 1.4142135623730951
Instance28 9.0 3.0 1.0
Instance12 8.0 2.0 1.0
Instance29 9.0 4.0 1.0
Instance30 9.0 5.0 1.0
Instance16 5.0 9.0 Infinity
Instance15 7.0 13.0 Infinity
Instance27 10.0 10.5 Infinity
Instance26 10.5 11.0 0.7071067811865476
Instance4 11.0 12.0 1.118033988749895
Instance5 11.0 13.0 1.0
Instance21 12.0 13.0 1.0
Instance6 13.0 13.0 1.0
Instance21 14.0 12.5 1.118033988749895
Instance22 14.5 11.5 1.118033988749895
Instance23 15.0 10.5 1.118033988749895
Instance24 15.0 9.5 1.0
Instance11 11.0 7.0 Infinity
Instance18 11.5 8.0 1.118033988749895
Instance7 12.0 8.5 0.7071067811865476
Instance25 12.0 9.5 1.0
Instance8 13.0 8.0 1.118033988749895
Instance9 13.0 9.0 1.0
Instance10 13.0 7.0 1.0
Instance20 13.0 10.0 1.0
Instance17 16.0 16.0 Infinity

The cluster ordering found by Optics can be used to do various things. Among others, it can be used to generate DBScan style clusters based on the density of points. This feature is implemented in SPMF and is called ExtractDBScanClusters() in the original paper presenting OPTICS. When extracting DBscan clusters it is possible to specify a different epsilon value than the one used to extract the cluster ordering. This new epsilon value is called "epsilonPrime" (see the paper for details). By extracting clusters with epsilonPrime =5, we can obtain three clusters.

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
[Instance1 1.0 1.0][Instance3 1.0 0.0][Instance2 0.0 1.0]
[Instance14 10.0 1.0][Instance13 9.0 2.0][Instance28 9.0 3.0][Instance12 8.0 2.0][Instance29 9.0 4.0][Instance30 9.0 5.0]
[Instance27 10.0 10.5][Instance26 10.5 11.0][Instance4 11.0 12.0][Instance5 11.0 13.0][Instance21 12.0 13.0][Instance6 13.0 13.0][Instance21 14.0 12.5][Instance22 14.5 11.5][Instance23 15.0 10.5][Instance24 15.0 9.5]
[Instance11 11.0 7.0][Instance18 11.5 8.0][Instance7 12.0 8.5][Instance25 12.0 9.5][Instance8 13.0 8.0][Instance9 13.0 9.0][Instance10 13.0 7.0][Instance20 13.0 10.0]

The output file format is defined as follows.The output file format is defined as follows. The first few lines indicate the attribute names. Each attribute is specified on a separated line with the keyword "ATTRIBUTEDEF=" followed by the attribute name (a string). Then, the list of clusters is indicated. Each cluster is specified on a separated line, listing the instances contained in the cluster. An instance is a name followed by a list of double values separated by " " and between the "[" and "]" characters.

The clusters found by the algorithm can be viewed visually using the "Cluster Viewer" provided in SPMF. If you are using the graphical interface of SPMF, click the checkbox "Cluster Viewer" before pressing the "Run Algorithm" button. The result will be displayed in the Cluster Viewer.

Note that it is normal that OPTICS may generate a cluster having less than minPts in some cases. Note also that OPTICS eliminate points that are seen as noise

By running OPTICS on the previous input file and minPts =2 and epsilon=5, we obtain the following output file containing 3 clusters:

Applying OPTICS to time series

Note that the OPTICS algorithm implementation in SPMF can also be applied to time series database such as the file contextSAX.txt in the SPMF distribution. To apply this algorithm to time series, it is necessary to set the "separator" parameter of this algorithm to "," since time series files separate values by "," instead of separating by spaces.

Where can I get more information about OPTICS?

OPTICS is a quite popular data mining algorithm. The original paper proposing this algorithm is:

Mihael Ankerst, Markus M. Breunig, Hans-Peter Kriegel, Jörg Sander (1999). OPTICS: Ordering Points To Identify the Clustering Structure. ACM SIGMOD international conference on Management of data. ACM Press. pp. 49–60.

Example 81 : Clustering Values with a Hierarchical Clustering algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "HierarchicalClustering" algorithm, (2) select the input file "inputDBScan2.txt", (3) set the output file name (e.g. "output.txt") (4) set maxDistance to 4 (5) click "Run algorithm".
    Note that there is also an optional parameter called "separator" if you want to use other types of input files where values are not separated by spaces. For example, for a file separated by ',', the parameter "separator" would have to be set to ",".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Hierarchical_clustering inputDBscan2.txt output.txt 4 euclidian in a folder containing spmf.jar and the example input file inputDBScan2.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestHierarchicalClustering.java" in the package ca.pfv.SPMF.tests.

What is this algorithm?

We have implemented a hierarchical clustering algorithm that is based on the description of Hierarchical Clustering Algorithms from
http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/hierarchical.html. The algorithm is used to separate a set of instances (vectors of double values each having a name) into groups of instances (clusters) according to their similarity. In this implementation the euclidean distance is used to compute the similarity.

The algorithm works as follow. It first create a cluster for each single instance (vector). Then it recursively try to merge clusters together to create larger clusters. To determine if two clusters can be merged, a constant "threshold" indicate the maximal distance between two clusters for merging.

In this implementation the user can choose between various distance functions to assess the similarity between vectors. SPMF offers the Euclidian distance, correlation distance, cosine distance, Manathan distance and Jaccard distance.

What is the input?

The input is a set of instances (each having a name and containinga vector of double values), a parameter "maxdistance" and a distance function.

The input file format is a text file containing several instances.

The first lines (optional) specify the name of the attributes used for describing the instances. In this example, two attributes will be used, named X and Y. But note that more than two attributes could be used. Each attribute is specified on a separated line by the keyword "@ATTRIBUTEDEF=", followed by the attribute name

Then, each instance is described by two lines. The first line (which is optional) contains the string "@NAME=" followed by the name of the instance. Then, the second line provides a list of double values separated by single spaces.

An example of input is provided in the file "inputDBScan2.txt" of the SPMF distribution. It contains 31 instances, each described by two attribute called X and Y.

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
@NAME=Instance1
1 1
@NAME=Instance2
0 1
@NAME=Instance3
1 0
@NAME=Instance4
11 12
@NAME=Instance5
11 13
@NAME=Instance6
13 13
@NAME=Instance7
12 8.5
@NAME=Instance8
13 8
@NAME=Instance9
13 9
@NAME=Instance10
13 7
@NAME=Instance11
11 7
@NAME=Instance12
8 2
@NAME=Instance13
9 2
@NAME=Instance14
10 1
@NAME=Instance15
7 13
@NAME=Instance16
5 9
@NAME=Instance17
16 16
@NAME=Instance18
11.5 8
@NAME=Instance20
13 10
@NAME=Instance21
12 13
@NAME=Instance21
14 12.5
@NAME=Instance22
14.5 11.5
@NAME=Instance23
15 10.5
@NAME=Instance24
15 9.5
@NAME=Instance25
12 9.5
@NAME=Instance26
10.5 11
@NAME=Instance27
10 10.5
@NAME=Instance28
9 3
@NAME=Instance29
9 4
@NAME=Instance30
9 5

For example, the first instance is named "Instance1", and it is a vector with two values: 1 and 1 for the attributes X and Y, respectively.

This input file represents a set of 2D points. But note that, it is possible to use more than two attributes to describe instances. To give a better idea of what the input file looks like, here is a visual representation:

The algorithm will group the instances according to their similarity. To do this, it is also necessary to specify the distance function to be used for comparing the instances. The distance function can be the Euclidian distance, correlation distance, cosine distance, Manathan distance and Jaccard distance. In the command line or GUI of SPMF, the distance function is specified by using one of these keywords: "euclidian", "correlation", "cosine", "manathan" and "jaccard" as parameter. In this example, the euclidian distance is used.

Furthermore, the user should also provide a parameter called maxDistance (a positive value > 0) to the algorithm. This parameter indicate the maximal distance allowed between the mean of two clusters to merge them into a single cluster.

What is the output?

The algorithm groups instances in clusters according to their similarity. In SPMF, the similarity is defined according to the distance function chosen by the user such as the Euclidian distance.

By running the algorithm on the previous input file with maxDistance = 4 , we can obtain the following output file containing 6 clusters:

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
[Instance1 1.0 1.0][Instance2 0.0 1.0][Instance3 1.0 0.0]
[Instance4 11.0 12.0][Instance5 11.0 13.0][Instance6 13.0 13.0][Instance21 12.0 13.0][Instance26 10.5 11.0][Instance27 10.0 10.5][Instance7 12.0 8.5][Instance18 11.5 8.0][Instance8 13.0 8.0][Instance9 13.0 9.0][Instance20 13.0 10.0][Instance25 12.0 9.5][Instance10 13.0 7.0][Instance11 11.0 7.0][Instance21 14.0 12.5][Instance22 14.5 11.5][Instance23 15.0 10.5][Instance24 15.0 9.5]
[Instance12 8.0 2.0][Instance13 9.0 2.0][Instance14 10.0 1.0][Instance28 9.0 3.0][Instance29 9.0 4.0][Instance30 9.0 5.0]
[Instance15 7.0 13.0]
[Instance16 5.0 9.0]
[Instance17 16.0 16.0]

The output file format is defined as follows. The first few lines indicate the attribute names. Each attribute is specified on a separated line with the keyword "ATTRIBUTEDEF=" followed by the attribute name (a string). Then, the list of clusters is indicated. Each cluster is specified on a separated line, listing the instances contained in the cluster. An instance is a name followed by a list of double values separated by " " and between the "[" and "]" characters.

The clusters found by the algorithm can be viewed visually using the "Cluster Viewer" provided in SPMF. If you are using the graphical interface of SPMF, click the checkbox "Cluster Viewer" before pressing the "Run Algorithm" button. The result will be displayed in the Cluster Viewer.

As it can be seen in this example, the result somewhat make sense. Points that are close to each other have put in the same clusters.

Applying this algorithm to time series

Note that this algorithm implementation in SPMF can also be applied to time series database such as the file contextSAX.txt in the SPMF distribution. To apply this algorithm to time series, it is necessary to set the "separator" parameter of this algorithm to "," since time series files separate values by "," instead of separating by spaces.

Where can I get more information about Hierarchical clustering?

There is a good introduction here:
http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/hierarchical.html.

Moreover, you could also read the free chapter on clustering from the book "introduction to data mining" by Tan, Steinbach and Kumar on the book website.

Example 82 : Visualizing clusters of points using the Cluster Viewer

How to run this example?

  • If you are using the graphical interface, (1) choose the "Vizualize_clusters_of_instances" algorithm, (2) select the input file "clustersDBScan.txt", (3) and click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Vizualize_clusters_of_instances clustersDBScan.txt
    in a folder containing spmf.jar and the example input file clustersDBScan.txt.
  • If you are using the source code version of SPMF, to run respectively this example, launch the file "MainTestClusterViewerFile.java"in the package ca.pfv.SPMF.tests.

What is the Cluster Viewer?

The Cluster Viewer is a tool offered in SPMF for visualizing a set of clusters using a chart. The Cluster Viewer provides some basic functions like zooming in, zooming out, printing, and saving the picture as an image. It is useful for visualizing the clusters found by clustering algorithms such as DBScan, K-Means and others.

What is the input of the Cluster Viewer?

The input is one or more clusters. A cluster is a list of instances. An instance is here a list of floating-point decimal numbers (a vector of double values).

Clusters are produced by clustering algorithms such as K-Means and DBScan. An example of clusters found by the DBScan algorithm is the following:

Cluster Data points
Cluster1 (1, 0), (1, 1), (0, 1)
Cluster2 (10, 10), (10, 13)(13, 13)
Cluster3 (54, 54), (57, 55) (55, 55)

This example set of clusters is provided in the file clustersDBScan.txt of the SPMF distribution.

What is the result of running the time series viewer?

Running theCluster Viewer will display the clusters visually. For example, for the above clusters, the clusters will be displayed as follows (note that this may vary depending on your version of SPMF).

 

Input file format

The input file format used by the cluster viewer is defined as follows. It is a text file.

The text file first defines the attributes used to describe the instances that have been clustered. An attribute is defined using the keyword "@ATTRIBUTEDEF=" followed by an attribute name, which is a string. Each attribute is defined on a separated line.

Then, the list of clusters is given, each cluster is specified on a separated line. For each cluster, the list of instances contained in the cluster is specified. An instance is a name followed by a list of double values separated by " " and between the "[" and "]" characters.

For instance, the input file for this example is the following:

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
[Instance2 1.0 0.0][Instance0 1.0 1.0][Instance1 0.0 1.0]
[Instance3 10.0 10.0][Instance4 10.0 13.0][Instance5 13.0 13.0]
[Instance6 54.0 54.0][Instance9 57.0 55.0][Instance7 55.0 55.0]

It indicates that there are two attributes named "X" and "Y" and that there are three clusters. The first cluster contains three instances: (1, 0), (1, 1) and (0, 1).

Implementation details

The Cluster Viewer has been implemented by reusing and extending some code provided by Yuriy Guskov under the MIT License for displaying charts.

Example 83 : Visualizing instances using the Instance Viewer

How to run this example?

  • If you are using the graphical interface, (1) choose the "Vizualize_clusters_of_instances" algorithm, (2) select the input file "inputDBScan2.txt", (3) and click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run Vizualize_clusters_of_instances inputDBScan2.txt
    in a folder containing spmf.jar and the example input file inputDBScan2.txt.
  • If you are using the source code version of SPMF, to run respectively this example, launch the file "MainTestInstanceViewerFile.java"in the package ca.pfv.SPMF.tests.

What is the Instance Viewer?

The Instance Viewer is a tool offered in SPMF for visualizing a set of instances used as input for clustering algorithms. The Instance Viewer provides some basic functions like zooming in, zooming out, printing, and saving the picture as an image. It is useful for visualizing the instances that will be given to a clustering algorithm as input. Visualizing instances can help to decide which algorithm should then be applied.

What is the input of the Instance Viewer?

The input is a file containing several instances. The input file format is defined as follows.

The first lines (optional) specify the name of the attributes used for describing the instances. In this example, two attributes will be used, named X and Y. But note that more than two attributes could be used. Each attribute is specified on a separated line by the keyword "@ATTRIBUTEDEF=", followed by the attribute name

Then, each instance is described by two lines. The first line (which is optional) contains the string "@NAME=" followed by the name of the instance. Then, the second line provides a list of double values separated by single spaces.

An example of input is provided in the file "inputDBScan2.txt" of the SPMF distribution. It contains 31 instances, each described by two attribute called X and Y.

@ATTRIBUTEDEF=X
@ATTRIBUTEDEF=Y
@NAME=Instance1
1 1
@NAME=Instance2
0 1
@NAME=Instance3
1 0
@NAME=Instance4
11 12
@NAME=Instance5
11 13
@NAME=Instance6
13 13
@NAME=Instance7
12 8.5
@NAME=Instance8
13 8
@NAME=Instance9
13 9
@NAME=Instance10
13 7
@NAME=Instance11
11 7
@NAME=Instance12
8 2
@NAME=Instance13
9 2
@NAME=Instance14
10 1
@NAME=Instance15
7 13
@NAME=Instance16
5 9
@NAME=Instance17
16 16
@NAME=Instance18
11.5 8
@NAME=Instance20
13 10
@NAME=Instance21
12 13
@NAME=Instance21
14 12.5
@NAME=Instance22
14.5 11.5
@NAME=Instance23
15 10.5
@NAME=Instance24
15 9.5
@NAME=Instance25
12 9.5
@NAME=Instance26
10.5 11
@NAME=Instance27
10 10.5
@NAME=Instance28
9 3
@NAME=Instance29
9 4
@NAME=Instance30
9 5

For example, the first instance is named "Instance1", and it is a vector with two values: 1 and 1 for the attributes X and Y, respectively.

What is the result of running theinstance viewer?

Running the Instance Viewer will display the instances visually. For example, for the above instances, the instances will be displayed as follows (note that this may vary depending on your version of SPMF).

 

Implementation details

The Instance Viewer has been implemented by reusing and extending some code provided by Yuriy Guskov under the MIT License for displaying charts.

Example 84 : Mining Frequent Sequential Patterns Using the PrefixSpan Algorithm

How to run this example?

To run the implementation of PrefixSpan by P. Fournier-Viger (PFV):

  • If you are using the graphical interface, (1) choose the "PrefixSpan" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run PrefixSpan contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestPrefixSpan_saveToMemory.java" in the package ca.pfv.SPMF.tests.

To run the version the implementation of PrefixSpan by A. Gomariz Peñalver (AGP):

  • If you are using the graphical interface, (1) choose the "PrefixSpan_AGP" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run PrefixSpan_AGP contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestPrefixSpan_AGP_saveToMemory.java" in the package ca.pfv.SPMF.tests

What is PrefixSpan?

PrefixSpan is an algorithm for discovering sequential patterns in sequence databases, proposed by Pei et al. (2001).

What is the input of PrefixSpan?

The input of PrefixSpan is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage).

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of distinct items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of PrefixSpan?

PrefixSpan discovers all frequent sequential patterns occurring in a sequence database (subsequences that occurs in more than minsup sequences of the database.

To explain more formally what is a sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

For example, if we run PrefixSpan with minsup= 50 % and with a maximum pattern length of 100 items, 53 sequential patterns are found. The list is too long to be presented here. An example of pattern found is "(1,2),(6)" which appears in the first and the third sequences (it has therefore a support of 50%). This pattern has a length of 3 because it contains three items. Another pattern is "(4), (3), (2)". It appears in the second and third sequence (it has thus a support of 50 %). It also has a length of 3 because it contains 3 items.

Optional parameter(s)

The PrefixSpan implementation allows to specify additional optional parameter(s) :

  • "maximum pattern length" allows to specify the maximum number of items that patterns found should contain.
  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 0 2" means that the pattern on this line appears in the first and the third sequences of the sequence database (sequences with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestPrefixSpan ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run PrefixSpan contextPrefixSpan.txt output.txt 50% 5 true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and patterns must have a maximum length of 5 items, and sequence ids should be output for each pattern found.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword "#SUP:" appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

2 3 -1 1 -1 #SUP: 2
6 -1 2 -1 #SUP: 2
6 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {2, 3}, followed by the itemset {1} has a support of 2 sequences.

Performance

PrefixSpan is one of the fastest sequential pattern mining algorithm. However, the SPAM and SPADE implementation in SPMF can be faster than PrefixSpan (see the "performance" section of the website for a performance comparison).

Implementation details

Note that in the source code, we also provide examples of how to keep the result into memory instead of saving it to a file. This can be useful if the algorithms are integrated into another Java software.

  • For the AGP version of PrefixSpan, the file MainTestPrefixSpan_AGP_saveToMemory shows how to run the algorithm and keep the result into memory.
  • For the PFV version, the file MainTestPrefixSpan_saveToMemory.java show how to run the algorithm to keep the result into memory.

Note also that in the source code, there is a version of PrefixSpan based on the PFV version that takes as input a dataset with strings instead of integers. It can be run by using the files MainTestPrefixSpan_WithStrings_saveToMemory.java and MainTestPrefixSpanWithStrings_saveToFile.java. For the graphical user interface version of SPMF, it is possible to use the version of PrefixSpan that uses Strings instead of integer by selecting "PrefixSpan with strings" and to test it with the input file contextPrefixSpanStrings.txt. The version of PrefixSpan with Strings was made to temporarily accommodate the needs of some users of SPMF. In the future, it may be replaced by a more general mechanism for using files with strings for all algorithms.

Where can I get more information about PrefixSpan?

The PrefixSpan algorithm is described in this article:

J. Pei, J. Han, B. Mortazavi-Asl, J. Wang, H. Pinto, Q. Chen, U. Dayal, M. Hsu: Mining Sequential Patterns by Pattern-Growth: The PrefixSpan Approach. IEEE Trans. Knowl. Data Eng. 16(11): 1424-1440 (2004)

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 85 : Mining Frequent Sequential Patterns Using the GSP Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "GSP" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50%, and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run GSP contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestGSP_saveToMemory.java" in the package ca.pfv.SPMF.tests.

What is GSP?

GSP is one of the first algorithm for discovering sequential patterns in sequence databases, proposed by Srikant et al. (1992). It uses an Apriori-like approach for discovering sequential patterns. Note that this version does not include the constraints proposed in the article.

What is the input of GSP?

The input of GSP is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage). Moreover, the implementation in SPMF adds another parameter, which is the maximum sequential pattern length in terms of items.

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of distinct items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of GSP?

GSP discovers all frequent sequential patterns occurring in a sequence database (subsequences that occurs in more than minsup sequences of the database.

To explain more formally what is a sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

For example, if we run GSP with minsup= 50 % and with a maximum pattern length of 100 items, 53 sequential patterns are found. The list is too long to be presented here. An example of pattern found is "(1,2),(6)" which appears in the first and the third sequences (it has therefore a support of 50%). This pattern has a length of 3 because it contains three items. Another pattern is "(4), (3), (2)". It appears in the second and third sequence (it has thus a support of 50 %). It also has a length of 3 because it contains 3 items.

Optional parameter(s)

The GSP implementation allows to specify additional optional parameter(s) :

  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 0 2" means that the pattern on this line appears in the first and the third sequences of the sequence database (sequences with ids 0 and 2).

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestGSP ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run GSP contextPrefixSpan.txt output.txt 50% 5 true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and sequence ids should be output for each pattern found

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

2 3 -1 1 -1 #SUP: 2
6 -1 2 -1 #SUP: 2
6 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {2, 3}, followed by the itemset {1} has a support of 2 sequences. The next lines follow the same format.

Performance

See the "performance" section of the website for a performance comparison with other sequential pattern mining algorithm.

Implementation details

The implementation is faithful to the article, except that the gap constraints and window constraints are currently not implemented (will be considered in future versions of SPMF).

Also note that in the source code, we also provide an example of how to run GSP and keep the result into memory instead of saving it to a file ("MainTestGSP_saveToMemory.java".). This can be useful if the algorithms are integrated into another Java software.

Where can I get more information about GSP?

The GSP algorithm is described in this article:

R. Srikant and R. Agrawal. 1996. Mining Sequential Patterns: Generalizations and Performance Improvements. In Proceedings of the 5th International Conference on Extending Database Technology: Advances in Database Technology (EDBT '96), Peter M. G. Apers, Mokrane Bouzeghoub, and Georges Gardarin (Eds.). Springer-Verlag, London, UK, UK, 3-17.

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 86: Mining Frequent Sequential Patterns Using the SPADE Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "SPADE" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run SPADE contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestSPADE_AGP_FatBitMap_saveToFile.java" in the package ca.pfv.SPMF.tests

What is SPADE?

SPADE is a popular sequential pattern mining algorithm proposed by Zaki

What is the input of SPADE?

The input of SPADE is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage). Moreover, the implementation in SPMF adds another parameter, which is the maximum sequential pattern length in terms of items.

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of distinct items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of SPADE?

SPADE discovers all frequent sequential patterns occurring in a sequence database (subsequences that occurs in more than minsup sequences of the database.

To explain more formally what is a sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

For example, if we run SPADE with minsup= 50 % and with a maximum pattern length of 100 items, 53 sequential patterns are found. The list is too long to be presented here. An example of pattern found is "(1,2),(6)" which appears in the first and the third sequences (it has therefore a support of 50%). This pattern has a length of 3 because it contains three items. Another pattern is "(4), (3), (2)". It appears in the second and third sequence (it has thus a support of 50 %). It also has a length of 3 because it contains 3 items.

Optional parameter(s)

The SPADE implementation allows to specify additional optional parameter(s) :

  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 1 3" means that the pattern on this line appears in the first and the third sequences of the sequence database.

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestSPADE ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run SPADE contextPrefixSpan.txt output.txt 50% true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and sequence ids should be output for each pattern found.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

2 3 -1 1 -1 #SUP: 2
6 -1 2 -1 #SUP: 2
6 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {2, 3}, followed by the itemset {1} has a support of 2 sequences. The next lines follow the same format.

Performance

See the "performance" section of the website for a performance comparison with other sequential pattern mining algorithm.

Implementation details

In the source code, we also provide examples of how to keep the result into memory instead of saving it to a file. This can be useful if the algorithms are integrated into another Java software. Examples of how to save result into memory are named according to the following naming convention: "MainTest..._saveToMemory".

Also note that in the source code, there are three variations the SPADE implementations that tries different ways to perform the join of IdLists. The fastest implementation is the one named "Fat_Bitmap". It is the one offered in the graphical user interface.

"MainTestSPADE_AGP_BitMap_saveToFile.java" "MainTestSPADE_AGP_BitMap_saveToMemory.java" "MainTestSPADE_AGP_EntryList_saveToFile.java" "MainTestSPADE_AGP_EntryList_saveToMemory.java" "MainTestSPADE_AGP_FatBitMap_saveToFile.java" "MainTestSPADE_AGP_FatBitMap_saveToMemory.java"

Lastly, in the source code, a parallelized version of SPADE is also offered:

"MainTestSPADE_AGP_Parallelized_BitMap_saveToFile.java" "MainTestSPADE_AGP_Parallelized_BitMap_saveToMemory.java" "MainTestSPADE_AGP_Parallelized_EntryList_saveToFile.java" "MainTestSPADE_AGP_Parallelized_EntryList_saveToMemory.java" "MainTestSPADE_AGP_Parallelized_FatBitMap_saveToFile.java" "MainTestSPADE_AGP_Parallelized_FatBitMap_saveToMemory.java"

Besides, note that an alternative input file contextSPADE.txt is provided. It contains the example used in the article proposing SPADE.

Where can I get more information about SPADE?

The SPADE algorithm is described in this article:

Mohammed J. Zaki. 2001. SPADE: An Efficient Algorithm for Mining Frequent Sequences. Mach. Learn. 42, 1-2 (January 2001), 31-60. DOI=10.1023/A:1007652502315 http://dx.doi.org/10.1023/A:1007652502315

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 87: Mining Frequent Sequential Patterns Using the CM-SPADE Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "CM-SPADE" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run CM-SPADE contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestCMSPADE_saveToFile.java" in the package ca.pfv.SPMF.tests

What is CM-SPADE?

CM-SPADE is a sequential pattern mining algorithm based on the SPADE algorithm.

The main difference is that CM-SPADE utilizes a new technique named co-occurrence pruning to prune the search space, which makes faster.

What is the input of CM-SPADE?

The input of CM-SPADE is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage). Moreover, the implementation in SPMF adds another parameter, which is the maximum sequential pattern length in terms of items.

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of distinct items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of CM-SPADE?

CM-SPADE discovers all frequent sequential patterns occurring in a sequence database (subsequences that occurs in more than minsup sequences of the database.

To explain more formally what is a sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

For example, if we run CM-SPADE with minsup= 50 % and with a maximum pattern length of 100 items, 53 sequential patterns are found. The list is too long to be presented here. An example of pattern found is "(1,2),(6)" which appears in the first and the third sequences (it has therefore a support of 50%). This pattern has a length of 3 because it contains three items. Another pattern is "(4), (3), (2)". It appears in the second and third sequence (it has thus a support of 50 %). It also has a length of 3 because it contains 3 items.

Optional parameter(s)

The CM-SPADE implementation allows to specify additional optional parameter(s) :

  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 1 3" means that the pattern on this line appears in the first and the third sequences of the sequence database.

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestCMSpade ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run CM-SPADE contextPrefixSpan.txt output.txt 50% true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and sequence ids should be output for each pattern found.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

2 3 -1 1 -1 #SUP: 2
6 -1 2 -1 #SUP: 2
6 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {2, 3}, followed by the itemset {1} has a support of 2 sequences. The next lines follow the same format.

Performance

CM-SPADE is faster than SPADE and also the best sequential pattern mining algorithm in SPMF according to our experiment in the CM-SPADE paper.

Implementation details

In the source code, we also provide examples of how to keep the result into memory instead of saving it to a file. This can be useful if the algorithms are integrated into another Java software. Examples of how to save result into memory is found in the following file: "MainTestCMSPADE_saveToMemory".

Where can I get more information about CM-SPADE?

The CM-SPADE algorithm is described in this article:

Fournier-Viger, P., Gomariz, A., Campos, M., Thomas, R. (2014). Fast Vertical Mining of Sequential Patterns Using Co-occurrence Information. Proc. 18th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2014), Part 1, Springer, LNAI, 8443. pp. 40-52.

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 88: Mining Frequent Sequential Patterns Using the SPAM Algorithm

How to run this example?

To run the implementation of SPAM by P. Fournier-Viger (PFV):

  • If you are using the graphical interface, (1) choose the "SPAM" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and maximum pattern length = 100, (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run SPAM contextPrefixSpan.txt output.txt 50% 100 in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestSPAM.java" in the package ca.pfv.SPMF.tests.

To run the version the implementation of SPAM by A. Gomariz Peñalver (AGP):

  • If you are using the graphical interface, (1) choose the "SPAM_AGP" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run SPAM_AGP contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestSPAM_AGP_FatBitMap_saveToFile.java" in the package ca.pfv.SPMF.tests. (other variations are also available in the source code)

What is SPAM?

SPAM is an algorithm for discovering frequent sequential patterns in a sequence database. It was proposed by Ayres (2002).

What is the input of SPAM?

The input of SPAM is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage). Moreover, the implementation in SPMF adds another parameter, which is the maximum sequential pattern length in terms of items.

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of SPAM?

SPAM discovers all frequent sequential patterns occurring in a sequence database (subsequences that occurs in more than minsup sequences of the database.

To explain more formally what is a sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

For example, if we run SPAM with minsup= 50 %, 53 sequential patterns will be found. The list is too long to be presented here. An example of pattern found is "(1,2),(6)" which appears in the first and the third sequences (it has therefore a support of 50%). This pattern has a length of 3 because it contains three items. Another pattern is "(4), (3), (2)". It appears in the second and third sequence (it has thus a support of 50 %). It also has a length of 3 because it contains 3 items.

Optional parameters

The SPAM implementation allows to specify four optional parameters :

  • "minimum pattern length" allows to specify the minimum number of items that patterns found should contain.
  • "maximum pattern length" allows to specify the maximum number of items that patterns found should contain.
  • "max gap" allows to specify if gaps are allowed in sequential patterns. For example, if "max gap" is set to 1, no gap is allowed (i.e. each consecutive itemset of a pattern must appear consecutively in a sequence). If "max gap" is set to N, a gap of N-1 itemsets is allowed between two consecutive itemsets of a pattern. If the parameter is not used, by default "max gap" is set to +∞.
  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 0 2" means that the pattern on this line appears in the first and the third sequences of the sequence database (sequences with ids 0 and 2).

These parameters are available in the GUI of SPMF and also in the example "MainTestSPAM.java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameters in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run SPAM contextPrefixSpan.txt output.txt 0.5 2 6 1 true
This command means to apply SPAM on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 0.5, that patterns must have a minimum length of 2 items, a maximum length of 6 items, and have no gap between itemsets, and that ids of sequence where the patterns is found must be shown in the output.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

2 3 -1 1 -1 #SUP: 2
6 -1 2 -1 #SUP: 2
6 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {2, 3}, followed by the itemset {1} has a support of 2 sequences. The next lines follow the same format.

Performance

SPAM is one of the fastest sequential pattern mining algorithm. The SPAM implementation in SPMF is reported to be faster than PrefixSpan (see the "performance" section of the website for a performance comparison). However, CM-SPAM is faster than SPAM.

Implementation details

In the source code, we also provide examples of how to keep the result into memory instead of saving it to a file. This can be useful if the algorithms are integrated into another Java software. Examples of how to save result into memory are named according to the following naming convention: "MainTest..._saveToMemory".

For the AGP implementation of SPAM, several version are provided in the source code that shows different way to perform the join of IdLists. The fastest implementation is the one named "Fat_Bitmap". It is the one offered in the graphical user interface.

  • MainTestSPAM_AGP_BitMap_saveToFile.java"
  • "MainTestSPAM_AGP_BitMap_saveToMemory.java"
  • "MainTestSPAM_AGP_EntryList_saveToFile.java"
  • "MainTestSPAM_AGP_EntryList_saveToMemory.java"
  • "MainTestSPAM_AGP_FatBitMap_saveToFile.java"
  • "MainTestSPAM_AGP_FatBitMap_saveToMemory.java"

The AGP and PFV implementations of SPAM shares some source code but also have some significant differences. See the performance section of the website for a performance comparison (will be added at the end of August 2013).

Where can I get more information about SPAM?

The SPAM algorithm was proposed in this paper:

J. Ayres, J. Gehrke, T.Yiu, and J. Flannick. Sequential Pattern Mining Using Bitmaps. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Edmonton, Alberta, Canada, July 2002.

The implementation of the optional "maxgap" constraint is based on this paper:

Ho, J., Lukov, L., & Chawla, S. (2005). Sequential pattern mining with constraints on large protein databases. In Proceedings of the 12th International Conference on Management of Data (COMAD) (pp. 89-100).

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 89: Mining Frequent Sequential Patterns Using the CM-SPAM Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "CM-SPAM" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run CMSPAM contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestCMSPAM_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is CMSPAM?

CM-SPAM (2013) is a sequential pattern mining algorithm based on the SPAM algorithm.

The main difference is that CM-SPAM utilizes a new technique named co-occurrence pruning to prune the search space, which makes it faster than the original SPAM algorithm.

What is the input of CM-SPAM?

The input of CM-SPAM is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage). Moreover, the implementation in SPMF adds another parameter, which is the maximum sequential pattern length in terms of items.

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of CM-SPAM?

CM-SPAM discovers all frequent sequential patterns occurring in a sequence database (subsequences that occurs in more than minsup sequences of the database.

To explain more formally what is a sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

For example, if we run CM-SPAM with minsup= 50 %, 53 sequential patterns will be found. The list is too long to be presented here. An example of pattern found is "(1,2),(6)" which appears in the first and the third sequences (it has therefore a support of 50%). This pattern has a length of 3 because it contains three items. Another pattern is "(4), (3), (2)". It appears in the second and third sequence (it has thus a support of 50 %). It also has a length of 3 because it contains 3 items.

Optional parameters

The CM-SPAM implementation allows to specify additional optional parameter(s) :

  • "minimum pattern length" allows to specify the minimum number of items that patterns found should contain.
  • "maximum pattern length" allows to specify the maximum number of items that patterns found should contain.
  • "required items" allow to specify a set of items that must appears in every patterns found.
  • "max gap" allows to specify if gaps are allowed in sequential patterns. For example, if "max gap" is set to 1, no gap is allowed (i.e. each consecutive itemset of a pattern must appear consecutively in a sequence). If "max gap" is set to N, a gap of N-1 itemsets is allowed between two consecutive itemsets of a pattern. If the parameter is not used, by default "max gap" is set to +∞.
  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 0 2" means that the pattern on this line appears in the first and the third sequences of the sequence database (sequences with ids 0 and 2).

These parameters are available in the GUI of SPMF and also in the example "MainTestCMSPAM.java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameters in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run CM-SPAM contextPrefixSpan.txt output.txt 0.5 2 6 1,3 1 true
This command means to apply CM-SPAM on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 0.5, and patterns must have a minimum length of 2 items, a maximum length of 6 items, must contain items 2 and 3, and have no gap between itemsets. Moreover, sequence ids should be output for each pattern found.

Now, let's say that you want to run the algorithm again with the same parameters except that you don't want to use the "required items" parameter. You could do as follows:
java -jar spmf.jar run CM-SPAM contextPrefixSpan.txt output.txt 0.5 2 6 "" 1 true

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

2 3 -1 1 -1 #SUP: 2
6 -1 2 -1 #SUP: 2
6 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {2, 3}, followed by the itemset {1} has a support of 2 sequences. The next lines follow the same format.

Performance

CM-SPAM is faster than SPAM and one of the best sequential pattern mining algorithm in SPMF according to our experiment in the CM-SPAM paper (see Performance section of the website for more details).

Where can I get more information about CM-SPAM?

The CM-SPAM algorithm is described in this article:

Fournier-Viger, P., Gomariz, A., Campos, M., Thomas, R. (2014). Fast Vertical Mining of Sequential Patterns Using Co-occurrence Information. Proc. 18th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2014), Part 1, Springer, LNAI, 8443. pp. 40-52.

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 90: Mining Frequent Sequential Patterns Using the LAPIN Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "LAPIN" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run LAPIN contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestLAPIN_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is LAPIN?

LAPIN (2005) is a sequential pattern mining algorithm based on the SPAM algorithm. It replaces join operations by border calculations (which are similar to a projected database) and uses a table called "Item-is-exist-table" to know if an item can appear after a given position in a sequence. There are several variations of LAPIN. In this implementation, we have followed the main one also known as LAPIN, LAPIN-SPAM and LAPIN-LCI, depending on the paper where it is described by the authors.

What is the input of LAPIN?

The input of LAPIN is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage). Moreover, the implementation in SPMF adds another parameter, which is the maximum sequential pattern length in terms of items.

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of LAPIN?

LAPIN discovers all frequent sequential patterns occurring in a sequence database (subsequences that occurs in more than minsup sequences of the database.

To explain more formally what is a sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

For example, if we run LAPIN with minsup= 50 %, 53 sequential patterns will be found. The list is too long to be presented here. An example of pattern found is "(1,2),(6)" which appears in the first and the third sequences (it has therefore a support of 50%). This pattern has a length of 3 because it contains three items. Another pattern is "(4), (3), (2)". It appears in the second and third sequence (it has thus a support of 50 %). It also has a length of 3 because it contains 3 items.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

2 3 -1 1 -1 #SUP: 2
6 -1 2 -1 #SUP: 2
6 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {2, 3}, followed by the itemset {1} has a support of 2 sequences. The next lines follow the same format.

Performance

LAPIN is quite a fast algorithm. However, it is most of the times slower than CM-SPADE and CM-SPAM on the datasets that we have compared. The implementation is quite optimized. Perhaps that additional optimizations could be found to improve the speed further.

Where can I get more information about LAPIN?

The LAPIN algorithm is described in this article:

Z. Yang, Y. Wang, and M. Kitsuregawa. LAPIN: Effective Sequential Pattern Mining Algorithms by Last Position Induction. Technical Report, Info. and Comm. Eng. Dept., Tokyo University, 2005.

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 91 : Mining Frequent Closed Sequential Patterns Using the ClaSP Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "ClaSP" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run ClaSP contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestClaSP_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is ClaSP?

ClaSP is a very efficient algorithm for discovering closed sequential patterns in sequence databases, proposed by Antonio Gomariz Peñalver et al. (2013). This implementation is the original implementation.

What is the input of ClaSP?

The input of ClaSP is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage).

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of ClaSP?

ClaSP discovers all frequent closed sequential patterns that occurs in a sequence database.

To explain more formally what is a closed sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

A closed sequential pattern is a sequential pattern such that it is not strictly included in another pattern having the same support.

Why using ClaSP? It can be shown that the set of closed sequential patterns is generally much smaller than the set of sequential patterns and that no information small. Moreover, finding closed sequential patterns is often much more efficient than discovering all patterns.

For example, if we run ClaSP with minsup= 50 % on the sequence database, the following patterns are found.

ID Closed Sequential Pattern Support
S1 (6) 75 %
S2 (5) 75 %
S3 (2), (3) 75 %
S4 (1), (2) 100 %
S5 (1), (3) 100 %
S6 (1 2), (6) 50 %
S7 (4), (3) 75 %
S8 (1) (2), (3) 50 %
S9 (1), (2 3), (1) 50 %
S10 (1), (3), (2) 75 %
S11 (1), (3), (3) 75 %
S12 (1 2), (4), (3) 50 %
S13 (6), (2), (3) 50 %
S14 (5), (2), (3) 50 %
S15 (4), (3), (2) 50 %
S16 (5), (6), (3), (2) 50 %
S17 (5), (1), (3), (2) 50 %

For instance, the sequential pattern "(1,2),(6)" appears in the first and third sequence (it has therefore a support of 50%). Another pattern is "(4), (3)". It appears in the second and third sequence (it has thus a support of 75 %).

Optional parameter(s)

The ClaSP implementation allows to specify additional optional parameter(s) :

  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 1 3" means that the pattern on this line appears in the first and the third sequences of the sequence database.

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestClaSP ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run ClaSP contextPrefixSpan.txt output.txt 50% true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and sequence ids should be output for each pattern found.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

1 2 -1 4 -1 3 -1 #SUP: 2
1 2 -1 6 -1 #SUP: 2
1 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {1, 2}, followed by the itemset {4}, followed by the itemset {3} has a support of 2 sequences. The next lines follow the same format.

Performance

ClaSP is a very efficient algorithm for closed sequential pattern mining. See the article proposing ClaSP for a performance comparison with CloSpan and SPADE. Note that CM-ClaSP is generally faster than ClaSP.

Implementation details

In the source code version of SPMF, there is also an example of how to use ClaSP and keep the result in memory instead of saving it to a file ( MainTestClaSP_saveToMemory.java ).

An alternative input file contextClaSP.txt is also provided. It contains the example sequence database used in the article proposing ClaSP.

Where can I get more information about this algorithm?

The ClaSP algorithm was proposed in this paper:

A. Gomariz, M. Campos, R. Marín and B. Goethals (2013), ClaSP: An Efficient Algorithm for Mining Frequent Closed Sequences. Proc. PAKDD 2013, pp. 50-61.

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 92 : Mining Frequent Closed Sequential Patterns Using the CM-ClaSP Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "CM-ClaSP" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run CM-ClaSP contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestCMClaSP_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is CM-ClaSP?

ClaSP is a very efficient algorithm for discovering closed sequential patterns in sequence databases, proposed (Gomariz et al, 2013).

CM-ClaSP is a modification of the original ClaSP algorithm using a technique co-occurrence pruning to prune the search space (Fournier-Viger, Gomariz et al., 2014). It is generally faster than the original ClaSP.

What is the input of CM-ClaSP?

The input of CM-ClaSP is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage).

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of CM-ClaSP?

CM-ClaSP discovers all frequent closed sequential patterns that occurs in a sequence database.

To explain more formally what is a closed sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

A closed sequential pattern is a sequential pattern such that it is not strictly included in another pattern having the same support.

Why using CM-ClaSP? It can be shown that the set of closed sequential patterns is generally much smaller than the set of sequential patterns and that no information small. Moreover, finding closed sequential patterns is often much more efficient than discovering all patterns.

For example, if we run CM-ClaSP with minsup= 50 % on the sequence database, the following patterns are found.

ID Closed Sequential Pattern Support
S1 (6) 75 %
S2 (5) 75 %
S3 (2), (3) 75 %
S4 (1), (2) 100 %
S5 (1), (3) 100 %
S6 (1 2), (6) 50 %
S7 (4), (3) 75 %
S8 (1) (2), (3) 50 %
S9 (1), (2 3), (1) 50 %
S10 (1), (3), (2) 75 %
S11 (1), (3), (3) 75 %
S12 (1 2), (4), (3) 50 %
S13 (6), (2), (3) 50 %
S14 (5), (2), (3) 50 %
S15 (4), (3), (2) 50 %
S16 (5), (6), (3), (2) 50 %
S17 (5), (1), (3), (2) 50 %

For instance, the sequential pattern "(1,2),(6)" appears in the first and third sequence (it has therefore a support of 50%). Another pattern is "(4), (3)". It appears in the second and third sequence (it has thus a support of 75 %).

Optional parameter(s)

The CM-ClaSP implementation allows to specify additional optional parameter(s) :

  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 1 3" means that the pattern on this line appears in the first and the third sequences of the sequence database.

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestCMClaSP ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run CM-ClaSP contextPrefixSpan.txt output.txt 50% true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and sequence ids should be output for each pattern found.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

1 2 -1 4 -1 3 -1 #SUP: 2
1 2 -1 6 -1 #SUP: 2
1 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {1, 2}, followed by the itemset {4}, followed by the itemset {3} has a support of 2 sequences. The next lines follow the same format.

Performance

ClaSP is a very efficient algorithm for closed sequential pattern mining. CM-ClaSP is generally a few times faster than ClaSP on most dataset (see the CM-ClaSP paper for details).

Implementation details

In the source code version of SPMF, there is also an example of how to use CM-ClaSP and keep the result in memory instead of saving it to a file ( MainTestCMClaSP_saveToMemory.java ).

Where can I get more information about this algorithm?

The CM-ClaSP algorithm was proposed in this paper:

Fournier-Viger, P., Gomariz, A., Campos, M., Thomas, R. (2014). Fast Vertical Mining of Sequential Patterns Using Co-occurrence Information. Proc. 18th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2014), Part 1, Springer, LNAI, 8443. pp. 40-52.

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 93 : Mining Frequent Closed Sequential Patterns Using the CloSpan Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "CloSpan" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run CloSpan contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestCloSpan_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is CloSpan?

CloSpan is a pattern-growth algorithm for discovering closed sequential patterns in sequence databases, proposed by Yan et al. (2003),

What is the input of CloSpan?

The input of CloSpan is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage).

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of CloSpan?

CloSpan discovers all frequent closed sequential patterns that occurs in a sequence database.

To explain more formally what is a closed sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

A closed sequential pattern is a sequential pattern such that it is not strictly included in another pattern having the same support.

Why using CloSpan? It can be shown that the set of closed sequential patterns is generally much smaller than the set of sequential patterns and that no information small. Moreover, finding closed sequential patterns is often much more efficient than discovering all patterns.

For example, if we run CloSpan with minsup= 50 % on the sequence database, the following patterns are found.

ID Closed Sequential Pattern Support
S1 (6) 75 %
S2 (5) 75 %
S3 (2), (3) 75 %
S4 (1), (2) 100 %
S5 (1), (3) 100 %
S6 (1 2), (6) 50 %
S7 (4), (3) 75 %
S8 (1) (2), (3) 50 %
S9 (1), (2 3), (1) 50 %
S10 (1), (3), (2) 75 %
S11 (1), (3), (3) 75 %
S12 (1 2), (4), (3) 50 %
S13 (6), (2), (3) 50 %
S14 (5), (2), (3) 50 %
S15 (4), (3), (2) 50 %
S16 (5), (6), (3), (2) 50 %
S17 (5), (1), (3), (2) 50 %

For instance, the sequential pattern "(1,2),(6)" appears in the first and third sequence (it has therefore a support of 50%). Another pattern is "(4), (3)". It appears in the second and third sequence (it has thus a support of 75 %).

Optional parameter(s)

The CloSpan implementation allows to specify additional optional parameter(s) :

  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 1 3" means that the pattern on this line appears in the first and the third sequences of the sequence database.

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestCloSpan ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run CloSpan contextPrefixSpan.txt output.txt 50% true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and sequence ids should be output for each pattern found.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

1 2 -1 4 -1 3 -1 #SUP: 2
1 2 -1 6 -1 #SUP: 2
1 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {1, 2}, followed by the itemset {4}, followed by the itemset {3} has a support of 2 sequences. The next lines follow the same format.

Performance

CloSpan is an efficient algorithm for closed sequential pattern mining. However, it should be noted that some newer algorithm like ClaSP have shown better performance on many datasets (see the ClaSP paper for a performance comparison).

Implementation details

In the source code version of SPMF, there is also an example of how to use ClaSP and keep the result in memory instead of saving it to a file ( MainTestCloSpan_saveToMemory.java ).

An alternative input file contextCloSpan.txt is also provided. It contains the example sequence database used in the article proposing CloSpan.

Where can I get more information about this algorithm?

The CloSpan algorithm was proposed in this paper:

Yan, X., Han, J., & Afshar, R. (2003, May). CloSpan: Mining closed sequential patterns in large datasets. In Proc. 2003 SIAM Int’l Conf. Data Mining (SDM’03) (pp. 166-177).

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 94 : Mining Frequent Closed Sequential Patterns Using the BIDE+ Algorithm

How to run this example?

  • If you are using the graphical interface, (1) choose the "BIDE+" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run BIDE+ contextPrefixSpan.txt output.txt 50% in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestBIDEPlus_saveToFile.java" in the package ca.pfv.SPMF.tests.

What is BIDE+?

BIDE+ is an algorithm for discovering closed sequential patterns in sequence databases, proposed by Wang et al.(2007).

What is the input of BIDE+?

The input of BIDE+ is a sequence database and a user-specified threshold named minsup (a value in [0,1] representing a percentage).

A sequence database is a set of sequences where each sequence is a list of itemsets. An itemset is an unordered set of items. For example, the table shown below contains four sequences. The first sequence, named S1, contains 5 itemsets. It means that item 1 was followed by items 1 2 and 3 at the same time, which were followed by 1 and 3, followed by 4, and followed by 3 and 6. It is assumed that items in an itemset are sorted in lexicographical order. This database is provided in the file "contextPrefixSpan.txt" of the SPMF distribution. Note that it is assumed that no items appear twice in the same itemset and that items in an itemset are lexically ordered.

ID Sequences
S1 (1), (1 2 3), (1 3), (4), (3 6)
S2 (1 4), (3), (2 3), (1 5)
S3 (5 6), (1 2), (4 6), (3), (2)
S4 (5), (7), (1 6), (3), (2), (3)

What is the output of BIDE+?

BIDE+ discovers all frequent closed sequential patterns that occurs in a sequence database.

To explain more formally what is a closed sequential pattern, it is necessary to review some definition.

A sequential pattern is a sequence. A sequence SA = X1, X2, ... Xk, where X1, X2... Xk are itemsets is said to occur in another sequence SB = Y1, Y2, ... Ym, where Y1, Y2... Ym are itemsets, if and only if there exists integers 1 <= i1 < i2... < ik <= m such that X1 ⊆ Yi1, X2 ⊆ Yi2, ... Xk ⊆ Yik.

The support of a sequential pattern is the number of sequences where the pattern occurs divided by the total number of sequences in the database.

A frequent sequential pattern is a sequential pattern having a support no less than the minsup parameter provided by the user.

A closed sequential pattern is a sequential pattern such that it is not strictly included in another pattern having the same support.

Why using BIDE+? It can be shown that the set of closed sequential patterns is generally much smaller than the set of sequential patterns and that no information small. Moreover, finding closed sequential patterns is often much more efficient than discovering all patterns.

For example, if we run BIDE+ with minsup= 50 % on the sequence database, the following patterns are found.

ID Closed Sequential Pattern Support
S1 (6) 75 %
S2 (5) 75 %
S3 (2), (3) 75 %
S4 (1), (2) 100 %
S5 (1), (3) 100 %
S6 (1 2), (6) 50 %
S7 (4), (3) 75 %
S8 (1) (2), (3) 50 %
S9 (1), (2 3), (1) 50 %
S10 (1), (3), (2) 75 %
S11 (1), (3), (3) 75 %
S12 (1 2), (4), (3) 50 %
S13 (6), (2), (3) 50 %
S14 (5), (2), (3) 50 %
S15 (4), (3), (2) 50 %
S16 (5), (6), (3), (2) 50 %
S17 (5), (1), (3), (2) 50 %

For instance, the sequential pattern "(1,2),(6)" appears in the first and third sequence (it has therefore a support of 50%). Another pattern is "(4), (3)". It appears in the second and third sequence (it has thus a support of 75 %).

Optional parameter(s)

The BIDE+ implementation allows to specify additional optional parameter(s) :

  • "show sequences ids?" (true/false) This parameter allows to specify that sequence ids of sequences containing a pattern should be output for each pattern found. For example, if the parameter is set to true, each pattern in the output file will be followed by the keyword #SID followed by a list of sequences ids (integers separated by space). For example, a line terminated by "#SID: 1 3" means that the pattern on this line appears in the second and the fourth sequences of the sequence database.

These parameter(s) are available in the GUI of SPMF and also in the example(s) "MainTestBIDEPlus ... .java" provided in the source code of SPMF.

The parameter(s) can be also used in the command line with the Jar file. If you want to use these optional parameter(s) in the command line, it can be done as follows. Consider this example:
java -jar spmf.jar run BIDE+ contextPrefixSpan.txt output.txt 50% true
This command means to apply the algorithm on the file "contextPrefixSpan.txt" and output the results to "output.txt". Moreover, it specifies that the user wants to find patterns for minsup = 50%, and sequence ids should be output for each pattern found.

Input file format

The input file format is defined as follows. It is a text file where each line represents a sequence from a sequence database. Each item from a sequence is a positive integer and items from the same itemset within a sequence are separated by single space. Note that it is assumed that items within a same itemset are sorted according to a total order and that no item can appear twice in the same itemset. The value "-1" indicates the end of an itemset. The value "-2" indicates the end of a sequence (it appears at the end of each line). For example, the input file "contextPrefixSpan.txt" contains the following four lines (four sequences).

1 -1 1 2 3 -1 1 3 -1 4 -1 3 6 -1 -2
1 4 -1 3 -1 2 3 -1 1 5 -1 -2
5 6 -1 1 2 -1 4 6 -1 3 -1 2 -1 -2
5 -1 7 -1 1 6 -1 3 -1 2 -1 3 -1 -2

The first line represents a sequence where the itemset {1} is followed by the itemset {1, 2, 3}, followed by the itemset {1, 3}, followed by the itemset {4}, followed by the itemset {3, 6}. The next lines follow the same format.

Note that it is also possible to use a text file containing a text (several sentences) if the text file has the ".text" extension, as an alternative to the default input format. If the algorithm is applied on a text file from the graphical interface or command line interface, the text file will be automatically converted to the SPMF format, by dividing the text into sentences separated by ".", "?" and "!", where each word is considered as an item. Note that when a text file is used as input of a data mining algorithm, the performance will be slightly less than if the native SPMF file format is used because a conversion of the input file will be automatically performed before launching the algorithm and the result will also have to be converted. This cost however should be small.

Output file format

The output file format is defined as follows. It is a text file. Each line is a frequent sequential pattern. Each item from a sequential pattern is a positive integer and items from the same itemset within a sequence are separated by single spaces. The value "-1" indicates the end of an itemset. On each line, the sequential pattern is first indicated. Then, the keyword " #SUP: " appears followed by an integer indicating the support of the pattern as a number of sequences. For example, a few lines from the output file from the previous example are shown below:

1 2 -1 4 -1 3 -1 #SUP: 2
1 2 -1 6 -1 #SUP: 2
1 -1 2 -1 3 -1 #SUP: 2

The first line indicates that the frequent sequential pattern consisting of the itemset {1, 2}, followed by the itemset {4}, followed by the itemset {3} has a support of 2 sequences. The next lines follow the same format.

Performance

BIDE+ is a very efficient algorithm for closed sequential pattern mining. This implementations includes all the optimizations described in the paper.

Implementation details

I have included three versions of BIDE+ in the SPMF distribution. The first one keeps the frequent itemsets into memory and print the results to the console (MainTestBIDEPlus_saveToMemory.java). The second one is a version that saves the result directly to a file (MainTestBIDEPlus_saveToFile.java). The second version is faster.

The third version of BIDE+ accepts strings instead of integers. It is available under the name "BIDE+ with strings" in the GUI version of SPMF or in the package ca.pfv.spmf.sequential_rules.bide_with_strings for the source code version of SPMF. To run it, you should use the input file: contextPrefixSpanStrings.txt.

Where can I get more information about this algorithm?

The BIDE algorithm is described in this paper:

J. Wang, J. Han: BIDE: Efficient Mining of Frequent Closed Sequences. ICDE 2004: 79-90

Besides, you may read this survey of sequential pattern mining, which gives an overview of sequential pattern mining algorithms.

Example 95 : Mining Frequent Closed Sequential Patterns by Post-Processing Using the PrefixSpan or SPAM Algorithm

What is this?

This example shows how to use the PrefixSpan and SPAM algorithms to discover all sequential patterns and keep only closed patterns by post-processing. This should be less efficient than using a dedicated algorithm for closed pattern mining like ClaSP, CloSpan and BIDE+.

How to run this example?

If you want to use SPAM with post-processing:

  • If you are using the graphical interface, (1) choose the "SPAM_PostProcessingClosed" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and maximum pattern length = 100, (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run SPAM_PostProcessingClosed contextPrefixSpan.txt output.txt 50% 100 in a folder containing spmf.jar and the example input file contextPrefixSpan.txt.
  • If you are using the source code version of SPMF, launch the file "MainTestSPAM_PostProcessingStepForClosedMining_saveToFile.java" in the package ca.pfv.SPMF.tests. (other variations are also available in the source code)

If you want to use PrefixSpan with post-processing:

  • If you are using the graphical interface, (1) choose the "PrefixSpan_PostProcessingClosed" algorithm, (2) select the input file "contextPrefixSpan.txt", (3) set the output file name (e.g. "output.txt") (4) set minsup = 50% and maximum pattern length to 100, (5) click "Run algorithm".
  • If you want to execute this example from the command line, then execute this command:
    java -jar spmf.jar run PrefixSpan_PostProcessingClosed contextPrefixSpan.txt output.txt 50% 100