Mining@home: towards a public-resource computing framework for distributed data mining

Lucchese C., Mastroianni C., Orlando S., Talia D.

Computational Theory and Mathematics  Desktop grids  Computer Science Applications  Theoretical Computer Science  Closed frequent itemsets  Software  Computer Networks and Communications  Data mining  Public-resource computing 

Several classes of scientific and commercial applications require the execution of a large number of independent tasks. One highly successful and low-cost mechanism for acquiring the necessary computing power for these applications is the 'public-resource computing', or 'desktop Grid' paradigm, which exploits the computational power of private computers. So far, this paradigm has not been applied to data mining applications for two main reasons. First, it is not straightforward to decompose a data mining algorithm into truly independent sub-tasks. Second, the large volume of the involved data makes it difficult to handle the communication costs of a parallel paradigm. This paper introduces a general framework for distributed data mining applications called Mining@home. In particular, we focus on one of the main data mining problems: the extraction of closed frequent itemsets from transactional databases. We show that it is possible to decompose this problem into independent tasks, which however need to share a large volume of the data. We thus introduce a data-intensive computing network, which adopts a P2P topology based on super peers with caching capabilities, aiming to support the dissemination of large amounts of information. Finally, we evaluate the execution of a pattern extraction task on such network.

Source: Concurrency and computation (Online) 22 (2010): 658–682. doi:10.1002/cpe.1545

Publisher: John Wiley & Sons,, Chichester, UK , Regno Unito

Metrics