Automatic Speech Recognition (ASR) in specialized domains like sensory science presents significant challenges, particularly for tonal languages such as Vietnamese. While state-of-the-art models like Whisper perform well on general tasks, they often fail to accurately capture critical, domain-specific terminology. To address this gap, this study introduces two key contributions: (1) SenSpeech-Coffee, a novel, curated dataset of Vietnamese sensory speech for coffee evaluation, and (2) SenWhisper, a family of models developed by fine-tuning Whisper and its Vietnamese-adapted variant, PhoWhisper, on this dataset. Our evaluation framework employs both the standard Word Error Rate (WER) and a domain-specific Term Error Rate (TER) to rigorously assess performance. The results demonstrate that our SenWhisper models significantly outperform both the original Whisper and the Vietnamese SOTA PhoWhisper baselines. The best-performing model, PhoWhisper-large-ft, achieved a WER of 0.2001 and an TER of 0.0274, showcasing a marked improvement in recognizing crucial sensory descriptors. Our analysis reveals that fine-tuning is essential for enhancing not just overall accuracy but, more importantly, semantic fidelity. This work provides a validated methodology, a robust ASR model, and a new public dataset, laying a strong foundation for applying ASR in sensory and consumer research within the Vietnamese context