Exploring clinical pathways: a novel BERTopic framework for dynamic analysis using temporal binning and semantic drift
摘要
Clinical pathways derived from electronic health records offer valuable insights into patient treatment behaviors and outcomes. While standard topic modeling techniques, such as Latent Dirichlet Allocation (LDA) models, are effective in extracting latent patterns from clinical data, they do not account for temporal evolution. Temporal topic modeling (TTM), a variant of LDA that incorporates temporal smoothing, captures sequential information but loses the semantic richness due to bag-of-words representation. To overcome these limitations, we present a novel BERTopic framework for temporal topic modeling that leverages contextual embeddings and temporal clustering to identify evolving patterns in patient encounters. Patient encounters were transformed into temporally ordered documents utilizing the Synthea electronic health records dataset and were compared against baseline models of LDA and temporal topic modeling. Evaluations were conducted using metrics such as topic coherence, normalized keyword quality at N, cosine and Jaccard similarity. Because of the extensive embedding computation and the fine-grained temporal binning, the proposed framework takes advantage of the parallel and distributed computing for the longitudinal EHR data. The results demonstrate that the proposed BERTopic framework achieves superior coherence and normalized keyword quality scores, yielding more interpretable and temporally consistent topics. These findings indicate efficient clinical pathway analysis for evidence-based decision-making.