Lifestyle-related diseases are a significant issue in modern society, and one of the proposed solutions involves intervention studies using messages generated by large language models (LLMs). In conventional research, the evaluation of these messages was typically performed by experts or through crowdsourcing. While such human feedback is crucial for improving message quality, it presents challenges in terms of time and cost. To address this, this study develops a model for predicting the effectiveness of behavior change messages, using LLM-based methods to reduce reliance on human evaluation. We collected survey data in which behavior change messages were manually evaluated, constructed a dataset from this data, and propose a method to predict their effectiveness using this dataset and LLMs. The proposed approach demonstrated an 11% improvement in prediction accuracy compared to traditional supervised learning methods, such as Random Forest.