Music Emotion Recognition is an important task in the field of music information retrieval, with applications in music generation, song recommendation, and playlist generation. However, given copyright concerns and the expense of human subject studies, researchers lack large datasets annotated with emotions. This dataset scarcity significantly limits the ability of researchers to train machine learning models that recognize emotion directly from audio. To address this need, we introduce a novel approach that leverages language models to generate emotion annotations. Specifically, we task ChatGPT to provide (1) numeric estimations (2) set of emotion words, and (3) long-form descriptions that characterize the emotive qualities of a piece of music. We consider 22,968 songs across five public datasets to facilitate a comparison of our ChatGPT synthetic annotations against human and previous algorithmic annotations. Although indirect, these annotations have the potential to provide insight into the emotional content of music, opening new possibilities for research and applications in the area of music emotion recognition.