Towards Efficient Collaborative Data Transmission in JointCloud: A Dynamic Chunking Mechanism
摘要
JointCloud computing, as a novel cross-cloud collaboration architecture, is dedicated to facilitating services around multi-cloud cooperation. The application of the decentralized and distributed features of IPFS within JointCloud offers a solution for data transmission challenges among cloud service providers, ensuring efficient and reliable transfer across diverse network environments. IPFS is a distributed file system, which has been widely used in recent years. However, IPFS still has shortcomings in some aspects. The fixed chunk design of files in IPFS cannot adapt to different file types, sizes and network conditions, which affects the performance of distributed transmission. In this paper, we propose a dynamic chunking mechanism in IPFS to optimize distributed data transmission, which can dynamically adjust the chunk size of each file according to various indicators of file and network environment. Firstly, our mechanism is able to automatically detect the information of files, nodes and networks in IPFS. Then, the optimal chunk size of each file can be calculated by summarizing and analyzing the information. Finally, all files are dynamically splited in terms of the chunk size. Compared with the current fixed chunking, the experimental results conducted on the IPFS private network illustrate that the dynamic chunking mechanism can significantly improve the distributed transmission performance.