A flexible analysis and prediction framework on resource usage in public clouds

In cloud computing environments, users can rent virtual machines (VMs) from cloud providers to execute their programs or provide network services. While using this kind of cloud services, one of the biggest problems for the users is to determine the proper number of VMs to complete the jobs considering both budget and time. In this paper, we propose a resource prediction framework (RPF), which can help users choose the minimum number of virtual machines to complete their jobs within a user specified time constraint. In order to verify the feasibility of RPF, we have done three case studies, namely parallel frequent pattern growth (FP-Growth), parallel K-means, and Particle Swarm Optimization (PSO). FP-growth, K-means and PSO are data intensive algorithms. These algorithms are typically executed repeatedly with different execution parameters to find the optimal results. When evaluating RPF by these algorithms in cloud environments, we have to modify them to parallel versions. The evaluation results indicate that RPF can successfully obtain the minimum number of VMs with acceptable errors. According to our case studies, the proposed RPF can be adopted by data intensive jobs by providing flexibility to both end users and cloud system providers.