In recent years, spiking neural networks (SNNs) have gained significant attention in visual recognition tasks due to the low computational energy. However, most SNNs have a large number of parameters, which limits their use on resource-limited devices. In this paper, we propose an Ensemble Binary Spiking Neural Network (EB-SNN) for accurate and memory-friendly visual recognition. The EB-SNN is modeled by Ensemble Binary Weights (EBW) module, which integrates multiple binary weights for lightweight SNN modeling. Meanwhile, we propose Knowledge Alignment Strategy to ensure that the EB-SNN can approximate a well-trained SNN for good performance. Experimental results show that the EB-SNN can achieve accuracy of 95.39% on CIFAR10, using \(9.3\%\) memory of full-precision SNN.