Data streams from non-stationary environments are inherently difficult for predictive techniques as data distributions may rapidly change (concept drifts), requiring self-evolving and efficient models. Another important challenge is learning from abundant and potentially useful unlabeled data that would be otherwise discarded by offline methods. In this work, we propose Semi-supervised Stochastic Weight Averaging (S3WA), a semi-supervised neural network that employs Stochastic Weight Averaging (SWA) to improve predictive performance over typical ensemble approaches. S3WA employs a Denoising Autoencoder to learn meaningful dynamic representations of data using both labeled and unlabeled instances. Unlike SWA, S3WA modifies the weight averaging strategy to continuously adapt to changes in the weight space and is capable of working in non-stationary environments. Our experiments confirmed that our approach can successfully learn from unlabeled data with a single model while delivering superior predictive performance than state-of-the-art techniques \(^1\) .