The substantial memory and computational requirements of large language models (LLMs) hinder their deployment on individual resource-constrained devices. This paper introduces HyperCluster, a framework for fully decentralized collaborative inference over peer-to-peer wireless networks. HyperCluster presents three core innovations: (1) a ring-based pipelined inference protocol where nodes deterministically self-organize into a computational ring based on device capabilities and pass intermediate states directly between peers via QUIC-based direct transport; (2) a generalizable model sharding methodology built on top of the Hugging Face Transformers library that automatically partitions any dense LLM across heterogeneous devices according to available memory; (3) selective layer loading from safetensors files, which only loads the tensor weights required by each node’s assigned shard. We validate HyperCluster on a heterogeneous cluster of consumer-grade devices, demonstrating distributed inference of models up to 3 billion parameters with comprehensive latency and throughput analysis across one to three node configurations.