Text-only training is a promising machine learning paradigm for training multimodal models without requiring data from every modality. However, despite the potential of text-only training, to date few studies have explored its use as an approximation of missing data for supervised learning in data-scarce environments. In this work we define the kinds of problems suited for text-only training and examine techniques to acquire or design text-based training data. We address the modality gap’s role in the performance of text-only training, and present a case study on classifying subjective audio timbre descriptions based on three kinds of text-only training data and six augmentation methods on eight audio-timbre datasets. Our work shows that the text-only training paradigm successfully trains audio classifiers without audio and opens the door to future work in examining the effectiveness of text-only training for supervised machine learning problems without available datasets.