This study introduces LeTMEMo, a framework that leverages unsupervised topic modeling to evaluate and enhance closed-vocabulary models and questionnaires. We use SERVQUAL as a case study, a service quality theoretical framework and dictionary that is widely used in social science and marketing research to quantify service quality concepts, including reliability, empathy, assurance, tangibles, and responsiveness. However, its fixed word lists and domain-general design limit its ability to capture the semantic patterns present in dynamic and context-specific corpora. Using LeTMEMo with BERTopic on the YelpCHI dataset, we examine how the discovered topics align with SERVQUAL dimensions across a broad set of embedding and clustering configurations. To assess lexical coverage and potential gaps, we compare two pipelines. One pipeline keeps only SERVQUAL terms, and the other removes only SERVQUAL terms while keeping the rest of the vocabulary intact. The results show that BERTopic, especially when paired with pretrained embeddings and topic-controlled clustering, generates coherent topics that align with SERVQUAL dimensions. The pipeline that excludes SERVQUAL terms reveals themes that fall outside the dictionary, suggesting opportunities for extensions and enhancements to SERVQUAL. LeTMEMo can potentially be applied to other closed-vocabulary models, such as the popular LIWC.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

LeTMEMo: Leveraging Topic Modeling for Evaluating (Closed-Vocabulary) Models

  • Vu Minh Hoang Dang,
  • Rakesh M. Verma

摘要

This study introduces LeTMEMo, a framework that leverages unsupervised topic modeling to evaluate and enhance closed-vocabulary models and questionnaires. We use SERVQUAL as a case study, a service quality theoretical framework and dictionary that is widely used in social science and marketing research to quantify service quality concepts, including reliability, empathy, assurance, tangibles, and responsiveness. However, its fixed word lists and domain-general design limit its ability to capture the semantic patterns present in dynamic and context-specific corpora. Using LeTMEMo with BERTopic on the YelpCHI dataset, we examine how the discovered topics align with SERVQUAL dimensions across a broad set of embedding and clustering configurations. To assess lexical coverage and potential gaps, we compare two pipelines. One pipeline keeps only SERVQUAL terms, and the other removes only SERVQUAL terms while keeping the rest of the vocabulary intact. The results show that BERTopic, especially when paired with pretrained embeddings and topic-controlled clustering, generates coherent topics that align with SERVQUAL dimensions. The pipeline that excludes SERVQUAL terms reveals themes that fall outside the dictionary, suggesting opportunities for extensions and enhancements to SERVQUAL. LeTMEMo can potentially be applied to other closed-vocabulary models, such as the popular LIWC.