Data generated by edge devices holds the potential to train intelligent autonomous systems across multiple domains like mineral exploration, geological surveying, and automated ore classification in mining operations. Often, these data are distributed and sensitive in nature due to their economic and strategic value, requiring robust security measures and ethical handling protocols to protect proprietary information and prevent misuse. Albeit the recent advances in distributed computing approaches like decentralised learning and federated learning to take advantage of this distributed data to train a model, there are security concerns about data leakage. To address the challenges of limited and sensitive data, we propose a novel data augmentation paradigm using an autoencoder. We apply this approach to data-scarce scenarios, specifically for classifying pixels in magnetic images. This approach generates synthetic data samples by exploiting the internal representations of the auto-encoder that preserve crucial information while minimizing the risk of data leakage, thereby overcoming the limitations of data scarcity and security concerns. We believe this approach represents a promising direction for training machine learning models with minimal data.