Disfluency labeling plays a crucial role in improving end-to-end automatic speech recognition (ASR) of spontaneous speech, which is often characterized by non-lexical and disfluent acoustic events such as hesitations, backchannels, and broken words. In this study, we revisit disfluency labeling methods and evaluate their effectiveness across three distinct Hungarian conversational speech datasets, including a novel corpus of spontaneous speech from patients with mental illness. Our experiments demonstrate that disfluency-aware modeling significantly reduces overall error rates across all datasets. Furthermore, training on mixed datasets—including both typical and atypical speech—yields substantial improvements in recognition accuracy. These findings indicate that robust recognition of both lexical content and disfluent elements is attainable, even in low-resource settings, provided that sufficient training data is available.