This paper presents the results of a research that created and analyzed a Multimedia dataset for building energy efficiency estimation. First a new Multimedia Building Energy Efficiency (MMBEE) dataset was created from publicly available data. This work then explored the use of the window-to-wall ratio (WWR) information from building facade images and integrated it with traditional tabular data to create new training data, in order to predict building energy efficiency measures. Finally, we discuss potential applications and future research directions in using the MMBEE dataset for building energy efficiency prediction. Throughout the paper, a number of important processes and analyses were performed, which include feature selection, data correlation analysis, WWR extraction, and comparison of deep network and random forest models in building energy efficiency estimation. From this first attempt at using the Multimedia dataset for building energy efficiency estimation, we found the performances of deep models were better than traditional models such as random forest. We also found that there was an optimal point of what features shall be used for the prediction. Nonetheless, the incorporation of the current WWR estimation results did not yield the anticipated enhancement in estimation performance. Subsequently, a comprehensive investigation was conducted to ascertain potential contributing factors, and several avenues for future research were identified to enhance the predictive utility of the WWR feature.