Complex event processing (CEP) is an essential technology for analyzing streams of events. A key feature of a modern CEP architecture is the ability to process both continuous queries and analytical ad hoc queries on high-volume streams. Both query types support common operations (filter, aggregation, joins) known in event stream and database systems. Additionally, a crucial and unique operation in CEP is pattern matching, which matches user-defined predicates to subsequences of events. We present our solution for a system supporting continuous queries, fast ingestion, and efficient analytical ad hoc queries. The system follows the principles of a Lambda Architecture and is specialized for a large variety of pattern-matching queries, including sequential, situation, and group patterns. To offer efficient processing, we use modern hardware in each of the components. For continuous queries, we explore multi-core CPUs and GPUs. For ingestion and ad hoc queries, we analyze SSDs and persistent memory as ways to provide a robust system. Furthermore, we explore unique characteristics of the hardware and event processing applications such as temporal data, energy efficiency, and compression. We give an overview of the overall systems, highlight the research accomplishments, and describe common application scenarios that benefit from our architecture.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Efficient Event Processing on Modern Hardware

  • Marius Kuhrt,
  • Nikolaus Glombiewski,
  • Michael Körber,
  • Andreas Morgen,
  • Dominik Brandenstein,
  • Bernhard Seeger

摘要

Complex event processing (CEP) is an essential technology for analyzing streams of events. A key feature of a modern CEP architecture is the ability to process both continuous queries and analytical ad hoc queries on high-volume streams. Both query types support common operations (filter, aggregation, joins) known in event stream and database systems. Additionally, a crucial and unique operation in CEP is pattern matching, which matches user-defined predicates to subsequences of events. We present our solution for a system supporting continuous queries, fast ingestion, and efficient analytical ad hoc queries. The system follows the principles of a Lambda Architecture and is specialized for a large variety of pattern-matching queries, including sequential, situation, and group patterns. To offer efficient processing, we use modern hardware in each of the components. For continuous queries, we explore multi-core CPUs and GPUs. For ingestion and ad hoc queries, we analyze SSDs and persistent memory as ways to provide a robust system. Furthermore, we explore unique characteristics of the hardware and event processing applications such as temporal data, energy efficiency, and compression. We give an overview of the overall systems, highlight the research accomplishments, and describe common application scenarios that benefit from our architecture.