Modern applications often require functionalities that single types of database systems cannot adequately fulfil, prompting the use of multiple types of databases for effective data management. Polyglot persistence enables the integration of various data storage technologies within a single application, leveraging the strengths of each system. This approach, however, introduces complexities in design and implementation, increasing programming complexity and necessitating knowledge of multiple languages. This article presents a practical method for achieving application-coordinated polyglot persistence across key-value, column-oriented, and document-oriented NoSQL databases, specifically focusing on Redis, Cassandra, and MongoDB, using Python. Our proposed three-layer architecture enhances data management and analytics by utilizing Jupyter Notebooks for interactive applications and employing Python libraries for orchestration, integrating specialized NoSQL databases. We encountered specific challenges, such as memory overload in Cassandra during complex query operations, which highlights the limitations in scalability under certain conditions. Additionally, our findings indicate significant variations in query execution times, with MongoDB demonstrating the fastest responses and Cassandra showing delays when handling complex queries. Using the SSB from TPC-H with a dataset of 6,000,000 entries, we evaluated loading times and query execution performance. This study aims to serve as a comprehensive guide for researchers and industry professionals seeking to implement a unified interface across heterogeneous NoSQL engines, providing insights into both the potentials and hurdles of polyglot persistence.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Polyglot Persistence in Heterogeneous NoSQL Database Systems with Python

  • Diana Martinez-Mosquera,
  • Rosa Navarrete,
  • Lorena Recalde,
  • Andres Andrade-Cabrera

摘要

Modern applications often require functionalities that single types of database systems cannot adequately fulfil, prompting the use of multiple types of databases for effective data management. Polyglot persistence enables the integration of various data storage technologies within a single application, leveraging the strengths of each system. This approach, however, introduces complexities in design and implementation, increasing programming complexity and necessitating knowledge of multiple languages. This article presents a practical method for achieving application-coordinated polyglot persistence across key-value, column-oriented, and document-oriented NoSQL databases, specifically focusing on Redis, Cassandra, and MongoDB, using Python. Our proposed three-layer architecture enhances data management and analytics by utilizing Jupyter Notebooks for interactive applications and employing Python libraries for orchestration, integrating specialized NoSQL databases. We encountered specific challenges, such as memory overload in Cassandra during complex query operations, which highlights the limitations in scalability under certain conditions. Additionally, our findings indicate significant variations in query execution times, with MongoDB demonstrating the fastest responses and Cassandra showing delays when handling complex queries. Using the SSB from TPC-H with a dataset of 6,000,000 entries, we evaluated loading times and query execution performance. This study aims to serve as a comprehensive guide for researchers and industry professionals seeking to implement a unified interface across heterogeneous NoSQL engines, providing insights into both the potentials and hurdles of polyglot persistence.