Objective: The aim of this study is to explore the energy expenditure patterns of elderly during walking activities within the community and to analyze the association between physical characteristics and walking energy expenditure. Methods: A cross-sectional study was conducted in two communities in Guangzhou, collecting walking activity data from 70 elderly. Using questionnaires, travel diaries, and map drawings, we recorded participants’ daily walking frequency, duration, and walking trajectories. The Pandolf equation was utilized to calculate the energy expenditure for a single walking trip. Pearson correlation analysis and ANOVA were employed to analyze the relationship between walking energy expenditure and physical characteristics as well as walking attributes. Results: The results showed that among elderly, walking energy expenditure was significantly associated not only with age (r = −0.919; p < 0.001), but also with the objective category of the activity (p = 0.007). Conclusion: The findings indicate that elderly with higher age have lower energy expenditure during walking. Additionally, the objective significantly affects walking energy expenditure. Future interventions in communities can promote an active and healthy lifestyle among elderly through various ways.