This study investigates the impact of incorporating environmental, social, and governance (ESG) linguistic features on the accuracy of financial predictions for large German companies. Utilizing advanced natural language processing techniques, particularly the MPNet model fine-tuned for ESG content, we analyze internal and external ESG documents to extract relevant topics and sentiments. Our proposed model combines these ESG linguistic features with traditional financial metrics to predict corporate profitability and capital structure. The results demonstrate that the integration of ESG linguistic features substantially improves prediction accuracy, outperforming models that rely solely on financial features or financial features combined with conventional ESG scores. Specifically, our model achieves the lowest mean absolute error and root mean squared error, along with the highest correlation coefficients in both prediction tasks. These findings highlight the value of ESG linguistic analysis in enhancing the predictive power of financial models, providing a more comprehensive assessment of corporate performance and sustainability.