Scholarlink

The scale of real estate market data has shown explosive growth, and traditional analysis methods have gradually exposed bottleneck problems such as poor real-time performance and low computing efficiency when processing real estate big data. To address these challenges, this study is dedicated to building a real estate big data real-time analysis system based on a distributed memory computing architecture. First, the data collection layer is built, which uses web crawler technology, database interfaces and other methods to collect various real estate data such as house prices, housing information, transaction data, etc. in real time from diversified channels such as real estate websites, government real estate transaction platforms, and real estate agency systems. At the same time, the collected data is preliminarily cleaned. Then, a distributed storage layer is built. This layer uses a combination of distributed file system Ceph and distributed database Cassandra to store the cleaned data in a distributed manner on multiple nodes. Next, the computing layer is designed, which is built on the distributed memory computing framework Apache Spark and contains multiple computing nodes. These nodes read data from the distributed storage layer and make full use of the high-speed read and write characteristics of in-memory computing to perform real-time processing and analysis of real estate data. Finally, the application layer is created to present the analysis results of the computing layer to real estate developers, investors and other users in the form of intuitive charts and reports. The system in this paper outperforms the traditional centralized architecture in terms of latency and throughput, with latency reduced to 200–497 ms and throughput increased to 5.97 MB/s. Through practical application cases, the system in this paper provides real estate developers and investors with accurate market trend analysis and investment decision support, improving the digital decision-making capabilities of the real estate industry.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Real Estate Big Data Real-Time Analysis System Based on Distributed Memory Computing Architecture

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Bo Yan

摘要

The scale of real estate market data has shown explosive growth, and traditional analysis methods have gradually exposed bottleneck problems such as poor real-time performance and low computing efficiency when processing real estate big data. To address these challenges, this study is dedicated to building a real estate big data real-time analysis system based on a distributed memory computing architecture. First, the data collection layer is built, which uses web crawler technology, database interfaces and other methods to collect various real estate data such as house prices, housing information, transaction data, etc. in real time from diversified channels such as real estate websites, government real estate transaction platforms, and real estate agency systems. At the same time, the collected data is preliminarily cleaned. Then, a distributed storage layer is built. This layer uses a combination of distributed file system Ceph and distributed database Cassandra to store the cleaned data in a distributed manner on multiple nodes. Next, the computing layer is designed, which is built on the distributed memory computing framework Apache Spark and contains multiple computing nodes. These nodes read data from the distributed storage layer and make full use of the high-speed read and write characteristics of in-memory computing to perform real-time processing and analysis of real estate data. Finally, the application layer is created to present the analysis results of the computing layer to real estate developers, investors and other users in the form of intuitive charts and reports. The system in this paper outperforms the traditional centralized architecture in terms of latency and throughput, with latency reduced to 200–497 ms and throughput increased to 5.97 MB/s. Through practical application cases, the system in this paper provides real estate developers and investors with accurate market trend analysis and investment decision support, improving the digital decision-making capabilities of the real estate industry.