SDindex: Distributed Trajectory Big Data Index Based on Spark
摘要
Trajectory big data contains a wealth of information, which has important application value in fields such as urban governance and nature conservation. However, this kind of data is usually characterized by large data volume, complex structure, and high processing difficulty. To cope with these challenges, distributed computing frameworks have become one of the solutions. Nevertheless, many existing distributed trajectory management systems have limited query support and lack query-specific optimization, resulting in inefficient processing and often do not support dynamic updating of data. In this paper, we propose a Spark-based distributed trajectory big data indexing technique (SDindex), which includes a global index and a local index: the upper layer of the global index is processed by time slicing, and the lower layer is partitioned by STR; the upper layer of the local index adopts linear order partitioning, and the lower layer enhances indexing performance by using R*-tree. This structure significantly improves the performance of spatio-temporal range queries and KNN queries. In addition, this paper also details the update algorithm of SDindex. Finally, this paper verifies the efficiency and feasibility of SDindex through comparative experiments.