A fast and low idle time method for mining frequent patterns in distributed and many-task computing environments

Chun-Cheng Lin, Sheng Hao Chung, Ju Chin Chen, Yuan Tse Yu, Kawuu W. Lin*

*此作品的通信作者

研究成果: Article同行評審

摘要

Association rules mining has attracted much attention among data mining topics because it has been successfully applied in various fields to find the association between purchased items by identifying frequent patterns (FPs). Currently, databases are huge, ranging in size from terabytes to petabytes. Although past studies can effectively discover FPs to deduce association rules, the execution efficiency is still a critical problem, particularly for big data. Progressive size working set (PSWS) and parallel FP-growth (PFP) are state-of-the-art methods that have been applied successfully to parallel and distributed computing technology to improve mining processing time in many-task computing, thereby bridging the gap between high-throughput and high-performance computing. However, such methods cannot mine before obtaining a complete FP-tree or the corresponding subdatabase, causing a high idle time for computing nodes. We propose a method that can begin mining when a small part of an FP-tree is received. The idle time of computing nodes can be reduced, and thus, the time required for mining can be reduced effectively. Through an empirical evaluation, the proposed method is shown to be faster than PSWS and PFP.

原文English
頁(從 - 到)613-641
頁數29
期刊Distributed and Parallel Databases
36
發行號4
DOIs
出版狀態Published - 1 12月 2018

指紋

深入研究「A fast and low idle time method for mining frequent patterns in distributed and many-task computing environments」主題。共同形成了獨特的指紋。

引用此