This article explored the application of natural language processing technology in the evaluation of new media advertising effectiveness, with a focus on the impact of user comment sentiment analysis on advertising effectiveness evaluation. The traditional methods for evaluating the effectiveness of new media advertising suffer from high data collection costs, accuracy, and objectivity. This article aims to address these issues. Research has found that analyzing comments and feedback posted by users on social media platforms, online forums, and other places, combined with natural language processing technology for sentiment analysis, can more accurately understand user emotional attitudes and opinions, providing accurate and comprehensive data support for advertisers. The user comment sentiment analysis technology combined with natural language processing improved evaluation efficiency and enabled rapid large-scale data analysis. This article utilized natural language processing technology to improve the accuracy and effectiveness of evaluating the effectiveness of new media advertising. In effectiveness evaluation, random forest had the highest efficiency in feature selection of text, with an average efficiency of 87.42%.