Scholarlink

Real-time forecasting systems face increasing demands for processing high-volume, high-dimensional, and heterogeneous data. Traditional relational databases struggle to meet the speed and scalability needs of modern analytics pipelines. This paper presents a hybrid storage architecture integrating SQL, NoSQL, vector, and graph databases to enhance data preprocessing and forecasting accuracy. The system incorporates adaptive query routing and dimensionality reduction techniques such as PCA, IPCA, SPCA, and Autoencoders to optimize feature engineering. Extensive benchmarking using synthetic multimodal datasets demonstrates significant improvements: query latency reduced by up to 59%, and forecasting accuracy enhanced by 27% (based on RMSE). The architecture also explores indexing trade-offs (HNSW, IVF) and scalable workload distribution strategies. Our findings support the application of hybrid storage models in real-time forecasting across sectors such as retail, energy, and healthcare.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Optimizing Forecasting Pipelines with Hybrid Storage Models for High-Speed Preprocessing

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Balasubramanian Panneerselvan,
  • Rakesh Ramakrishnan,
  • Sandeep Shankar Harikrishnan

摘要

Real-time forecasting systems face increasing demands for processing high-volume, high-dimensional, and heterogeneous data. Traditional relational databases struggle to meet the speed and scalability needs of modern analytics pipelines. This paper presents a hybrid storage architecture integrating SQL, NoSQL, vector, and graph databases to enhance data preprocessing and forecasting accuracy. The system incorporates adaptive query routing and dimensionality reduction techniques such as PCA, IPCA, SPCA, and Autoencoders to optimize feature engineering. Extensive benchmarking using synthetic multimodal datasets demonstrates significant improvements: query latency reduced by up to 59%, and forecasting accuracy enhanced by 27% (based on RMSE). The architecture also explores indexing trade-offs (HNSW, IVF) and scalable workload distribution strategies. Our findings support the application of hybrid storage models in real-time forecasting across sectors such as retail, energy, and healthcare.