Size Effect of Pre-trained Models on Missing Value Imputation for Multi-criteria Recommender Systems
摘要
Recommender systems, especially multi-criteria collaborative filtering (MCCF) techniques, are most benefited by evaluations across a multitude of sub-criteria. Nevertheless, increasing the number of criteria leads to more missing rating values. In this study, we examine the effects of the size of pre-trained language models on missing-value imputation and recommendation performance in MCCF systems. We compare three transformer variants, BERT-Tiny (4.4M parameters), TinyBERT (14.5M), and BERT-base (110M), on the BeerAdvocate dataset. Each model is fine-tuned to predict missing criterion ratings from review text, and the enriched dataset feeds a cosine-based MCCF engine for Top-N recommendations. Results reveal that smaller models require substantially less training time while achieving comparable validation loss, yielding competitive coverage, precision, recall, and nDCG against the BERT-base. These findings demonstrate that lightweight transformers offer an effective, resource-efficient approach to mitigate data sparsity in MCCF systems.