AI-powered Machine Transaltion (MT) services have improved substantially in recent years, with their adoption by a wide spectrum of users, ranging from casual, everyday needs to more complex, high-risk settings. However, there is still a need for continuous quality assessments, supported by user feedback to better understand end-user experiences with this technology. Our study presents the first large-scale investigation into user perceptions of these services by analysing their feedback on social media and app review platforms. We introduce a new benchmark, UEwMT, constructed from a comprehensive collection of online user-generated content, presenting a valuable resource for the wider MT and ML research communities. Using deep learning-based sentiment analysis and topic modelling, we systematically provide assessment of MT services from the user’s perspective, enabling the identification of meaningful usage patterns in real-world contexts. In-depth data-driven studies are crucial for MT providers to make informed improvements, especially with previous studies mainly using subjective assessments, such as surveys. The findings reveal distinct sentiment differences between social media users and app platform users; the former often expressing critical views, while the latter offering more constructive feedback. Furthermore, user confidence in these services is still statistically not high, and satisfaction only increases temporarily with updates or new features, which simultaneously leads to greater dissatisfaction if and when they fail to meet user expectations or simply match previous experiences. These insights allow the identification of key areas for feature enhancement, guiding the development of more effective and user-friendly MT solutions.