A Framework for Simulating Real-world Stream Data of the Internet of Things

With the rapid growth in the number of devices of the Internet of Things (IoT), the volume and types of stream data are rapidly increasing in the real world. Unfortunately, the stream data has the characteristics of infinite and periodic volatility in the real world, which cause problems with the inefficient stream processing tasks. In this study, we report our recent efforts on this issue, with a focus on simulating stream data. Firstly, we explore the characteristics of the real-world stream data of the IoT, which helps us to understand the stream data in the real world. Secondly, the pipeline of simulating stream data is proposed, which can accurately and efficiently simulate the characteristics of the stream data to improve efficiency for specific tasks. Finally, we design and implement a novel framework that can simulate various stream data for related stream processing tasks. To verify the validity of the proposed framework, we apply this framework to stream processing task running in the stream processing system. The experimental results reveal that the related stream processing task is accelerated by at least 24 times using our proposed simulation framework with the premise of ensuring volatility and trends of stream data.