While reward models in RLHF provide effective text generation evaluation, they lack interpretability, creating challenges for model optimization and error analysis. This paper introduces SHAP-RM, a framework that adapts Shapley Additive Explanations to reward model evaluation, enabling token-level interpretability for text generation quality assessment in role playing dialogue scenarios. Our approach treats reward models as black-box functions and applies SHAP’s game-theoretic foundation to decompose continuous quality scores into additive token contributions. We develop a standardized methodology for cross-model comparison that reveals model-specific strengths and weaknesses through attribution analysis, specifically targeting the complex evaluation requirements of character-driven dialogue generation. We demonstrate the SHAP-RM framework through two comprehensive experiments: controlled text perturbation to validate quality attribution capabilities, and systematic cross-model dialogue generation comparison using structured role-playing scenarios. The methodology successfully identifies quality-relevant content, distinguishes between different generation strategies, and provides interpretable insights into different models’ evaluation patterns for character consistency and dialogue coherence. SHAP-RM transforms opaque reward scores into actionable token-level explanations, offering a practical solution for understanding generation quality assessment in role-playing dialogue systems and enabling more informed model development decisions.