This paper explores the application of Variational Autoencoders (VAEs) to sparse non-visual data, focusing on a case study involving proximity sensors on a mobile robot. Traditionally effective in dense, high-dimensional domains like image processing, VAEs face unique challenges when adapted to sparse, low-dimensional sensory data. This paper contributes to broader efforts to tailor deep generative models to the complexities of robotic sensory data, offering insights that could enhance machine perception in robotic applications. Our study demonstrates that conventional VAE settings, particularly the weighting of the KL divergence ( \(\beta \) = 1), lead to suboptimal sparse representations for proximity sensor data with limited expressivity of learned latent space in the proposed case study of a mobile robot with proximity sensors. By systematically adjusting the \(\beta \) parameter and evaluating reconstruction quality and latent space utilization, we find that lower \(\beta \) values yield better results in sparse data scenarios. These findings suggest that adaptive or dynamic approaches to setting model parameters may be necessary to optimize VAE performance across varying data types, or finding alternative reconstruction loss formulations.