<p>Nanopore direct RNA sequencing has enabled the detection of RNA modifications in native RNA molecules, initially through the analysis of signal alterations and base-calling errors. More recently, modification prediction has been integrated into the base-calling step using pretrained, modification-aware base-calling models. So far, such models have been made available for <i>N</i>⁶-methyladenosine&#xa0;(m⁶A), inosine&#xa0;(I), pseudouridine (Ψ) and <i>N</i>⁵-methylcytosine&#xa0;(m⁵C), enabling RNA modification mapping in single-molecule resolution. However, their performance remains largely unclear. In this Progress, we discuss key limitations and uncertainties associated with base-calling models, including their potential cross-reactivities with other modifications, variability in false positive rates across models, unclear threshold choices for modification calling, insufficient orthogonal validation of model accuracy and lack of standardized analysis pipelines. To illustrate some of these issues, we compared the performance of three base-calling models on identical RNA samples, observing over 20-fold differences in the number of predicted m⁶A-modified sites. As these models are increasingly adopted, it is crucial to understand their limitations to ensure best practices and avoid misinterpretation of epitranscriptomics data.</p>