CSCE: a crawler engine for Cloud services discovery on the world wide web

Abstract

Over the past few years, Cloud computing has been receiving much attention as a new computing paradigm for providing flexible and on-demand infrastructures, platforms and software as services. In Cloud computing, challenges in searching cloud services need to be renewed due to a number of unique characteristics of cloud services such as the dynamic, diverse services offering at different levels, as well as the lack of standardized description languages. In this paper, we present the Cloud Service Crawler Engine that we used to collect metadata of 5, 883 valid cloud services through search engines after parsing more than a half million possible links. Based on the collected data, we conducted a set of statistical analysis and present the results in this paper. These statistical results offer an overall view on the current status of cloud services. Some most intriguing findings from our investigation include: i) the scarcity of standardization in Cloud computing, and ii) little evidence on a strong support of Cloud computing from the established service-oriented computing (SOC) technologies. Our findings provide some further insights on improving cloud services discovery and the datasets collected from this study will be valuable to the research community.