Aquaculture is a key contributor to Aotearoa New Zealand’s economy, with greenshell mussels representing a major export. As farms move further offshore into more demanding conditions, maintaining float line buoyancy becomes increasingly challenging. Buoyancy failures can lead to mussel loss, highlighting the need for automated, scalable monitoring. Our deep learning-based buoyancy estimation approach offers a cost-effective, automated solution for aquaculture maintenance. Saliency map analyses reveal that neighbouring floats serve as contextual cues, providing motivation for the proposed novel context-aware strategy by considering scale of the context region relative to the target float for image-based assessment. By expanding each float’s background context to include neighbouring floats, we achieve a 7–8% increase in balanced accuracy across ResNet-18, DenseNet-169, and Vision Transformer models. An ablation study confirms that leveraging the right amount of surrounding context resolves the trade-off between useful signal and background noise. These findings demonstrate that our context-aware method can generalise well for both CNN and Transformer-based benchmarks. It can yield substantial performance gains in mussel float buoyancy monitoring.