Facial Expression Recognition (FER) is a fundamental task in computer vision, yet models often suffer from poor generalization to unseen data. The CAFE (Cognition of humAns for Facial Expression) model represents a landmark approach to this challenge, learning a mask to select expression-relevant features from a fixed, general-purpose backbone like CLIP. While effective, this raises the question of whether its masking strategy is optimal. To investigate this question, we conduct an exploratory study of three different feature aggregation strategies for CAFE: (1) CAFE-SR, which incorporates a sparsity regularization term to learn a more concise mask; (2) CAFE-Fuse, which replaces masking with an adaptive fusion of features from fixed and trainable backbones; and (3) CAFE-Align, which enhances fusion with cross-attention and style alignment. Our extensive experiments across five FER datasets show that our proposed strategies, particularly CAFE-SR and CAFE-Fuse, achieve superior generalization performance in the majority of training scenarios, suggesting that CAFE original masking mechanism is not the definitive solution for FER domain generalization.