Event-driven stream processing has emerged as a fundamental paradigm for real-time data analysis in various domains, ranging from financial markets to IoT applications. Various stream processing frameworks have been developed in response to this demand. Among these, event-driven stream processing is critical because it continuously consumes incoming data stream while returning query results only when particular events occur. However, in the conventional query execution approach, even in the absence of relevant events, incoming data instances are continually filtered by queries, even though they do not yield any useful insights. Resultantly, it not only increases the system's computational burden but also wastes valuable system resources. To address this challenge, an intelligent event-driven stream processing approach that uses sliding window concept to buffer incoming data instances in key instances has been developed. The queued data instances are released and executed by ensuing operators when an event occurs. When certain queued instances within the window become obsolete owing to limited size of the window prior to the event occurrence, these immediately disappear from the window. As a result, system overhead decreases while throughput increases. Extensive CQL experiments have been conducted to validate the effectiveness of the proposed approach on Auction and Bidding Data streams.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Event-Driven Stream Processing Using Window-Based Query Execution

  • Shruti Arora,
  • Anushka Aggarwal,
  • Rinkle Rani

摘要

Event-driven stream processing has emerged as a fundamental paradigm for real-time data analysis in various domains, ranging from financial markets to IoT applications. Various stream processing frameworks have been developed in response to this demand. Among these, event-driven stream processing is critical because it continuously consumes incoming data stream while returning query results only when particular events occur. However, in the conventional query execution approach, even in the absence of relevant events, incoming data instances are continually filtered by queries, even though they do not yield any useful insights. Resultantly, it not only increases the system's computational burden but also wastes valuable system resources. To address this challenge, an intelligent event-driven stream processing approach that uses sliding window concept to buffer incoming data instances in key instances has been developed. The queued data instances are released and executed by ensuing operators when an event occurs. When certain queued instances within the window become obsolete owing to limited size of the window prior to the event occurrence, these immediately disappear from the window. As a result, system overhead decreases while throughput increases. Extensive CQL experiments have been conducted to validate the effectiveness of the proposed approach on Auction and Bidding Data streams.