Recently, pretrained models have achieved high accuracy in Environmental Sound Classification (ESC). However, these models often neglect critical efficiency factors, such as training cost and model size, which are essential for practical deployment. To address this issue, we propose an efficient solution for general ESC tasks that leverages cross-domain transfer learning and ensemble learning. Our approach adapts pretrained compact image classifiers to ESC datasets and combines these well-adapted models using ensemble strategies based on efficiency metrics, ensuring both robustness and accuracy. Experimental results demonstrate that our method not only reduces model size and training cost but also achieves superior performance compared to individual models, highlighting its potential for practical applications in resource-constrained environments.