Strainer: Windowing-Based Advanced Sampling in Stream Processing Systems
摘要
Stream Processing is a very effective predominant paradigm for data processing. It provides an efficient approach to extract information from new data, as the data arrives. However, spikes in data throughput, can impact the accuracy and latency guarantees stream processing systems provide. This work proposes data sampling, a type of data reduction, as a solution to this problem. It provides a user-transparent implementation of two sampling methods in the Apache Spark Streaming framework. The results show a reduced amount of input data, leading to decreased processing time, but retaining a good accuracy in the extracted information.