We provide a new benchmark, called Sparqloscope, for evaluating the query performance of SPARQL engines. The benchmark combines three unique features, which separates it from other benchmarks: 1. Sparqloscope is generic in that it can be applied to any given RDF dataset and it will then produce a comprehensive benchmark for that particular dataset. Existing benchmarks are either synthetic, designed for a fixed dataset, or require a query log. 2. Sparqloscope is comprehensive in that it considers most features of the SPARQL 1.1 query language that are relevant in practice. In particular, it considers advanced features like EXISTS, and various SPARQL functions for numerical values, strings, dates, language filters, etc. 3. Sparqloscope is specific in that it aims to evaluate relevant features in isolation and as concisely as possible. The benchmark generated for a given knowledge graph consists of only around 100 very carefully crafted queries, the results of which can and should be studied individually and not in aggregation. Sparqloscope is free and open-source software and easy to use. As a showcase, we use it to evaluate the performance of six SPARQL engines (QLever, Virtuoso, MillenniumDB, GraphDB, Blazegraph, Jena) on two widely used RDF datasets (DBLP and Wikidata). The full materials and more are provided on https://purl.org/ad-freiburg/sparqloscope .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Sparqloscope: A Generic Benchmark for the Comprehensive and Concise Performance Evaluation of SPARQL Engines

  • Hannah Bast,
  • Johannes Kalmbach,
  • Robin Textor-Falconi,
  • Christoph Ullinger

摘要

We provide a new benchmark, called Sparqloscope, for evaluating the query performance of SPARQL engines. The benchmark combines three unique features, which separates it from other benchmarks: 1. Sparqloscope is generic in that it can be applied to any given RDF dataset and it will then produce a comprehensive benchmark for that particular dataset. Existing benchmarks are either synthetic, designed for a fixed dataset, or require a query log. 2. Sparqloscope is comprehensive in that it considers most features of the SPARQL 1.1 query language that are relevant in practice. In particular, it considers advanced features like EXISTS, and various SPARQL functions for numerical values, strings, dates, language filters, etc. 3. Sparqloscope is specific in that it aims to evaluate relevant features in isolation and as concisely as possible. The benchmark generated for a given knowledge graph consists of only around 100 very carefully crafted queries, the results of which can and should be studied individually and not in aggregation. Sparqloscope is free and open-source software and easy to use. As a showcase, we use it to evaluate the performance of six SPARQL engines (QLever, Virtuoso, MillenniumDB, GraphDB, Blazegraph, Jena) on two widely used RDF datasets (DBLP and Wikidata). The full materials and more are provided on https://purl.org/ad-freiburg/sparqloscope .