A sketch is a probabilistic data-structure useful for collecting statistics about data streams, such as the frequency of each item. Sketches require sub-linear memory and can be updated and queried quickly. Their estimation errors are bounded to a known range with a certain probability, both determined by the sketch’s parameters, which allows users to tailor a sketch to their accuracy needs. Conventional sketches, such as count-sketch (CS) and count-min sketch (CMS) are configured with a fixed size, set during their initialization. That is, their parameters remain constant, while their absolute estimation error grows with the length of the stream. Unfortunately, this means that additional memory cannot be utilized to limit the estimation error buildup. Being able to dynamically increase and decrease the sketch’s memory usage is therefore attractive when the stream size is unknown in advance, or memory availability changes after initialization. To that end, we present Geometric Sketch (GS), a novel sketch that supports both increasing and decreasing memory usage at a granularity of a single counter, with a memory overhead of only 2 integers that are not counters. All of our code is open sourced [2].

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Geometric Sketch: The Inflatable-Shrinkable Sketch

  • Dvir Biton,
  • Roy Friedman,
  • Rana Shahout

摘要

A sketch is a probabilistic data-structure useful for collecting statistics about data streams, such as the frequency of each item. Sketches require sub-linear memory and can be updated and queried quickly. Their estimation errors are bounded to a known range with a certain probability, both determined by the sketch’s parameters, which allows users to tailor a sketch to their accuracy needs. Conventional sketches, such as count-sketch (CS) and count-min sketch (CMS) are configured with a fixed size, set during their initialization. That is, their parameters remain constant, while their absolute estimation error grows with the length of the stream. Unfortunately, this means that additional memory cannot be utilized to limit the estimation error buildup. Being able to dynamically increase and decrease the sketch’s memory usage is therefore attractive when the stream size is unknown in advance, or memory availability changes after initialization. To that end, we present Geometric Sketch (GS), a novel sketch that supports both increasing and decreasing memory usage at a granularity of a single counter, with a memory overhead of only 2 integers that are not counters. All of our code is open sourced [2].