Predicting the internal conditions of people who have difficulties explaining their internal states because of dementia or other problems has gained increasing attention. Monitoring internal states requires precise evaluation of weak (less intense) facial expressions, which is challenging due to the significant influence of individual characteristics on facial expressions. For instance, a person may look more like they are smiling or frowning than others even when they are in a neutral state. In addition, ambiguous changes may occur. For example, subtle changes from neutral to positive facial expressions could be similar to those from neutral to negative expressions, which further complicates the evaluation of facial expressions. Conventional FER systems designed for various people with various personal differences often have difficulties in detecting and indexing such subtle changes caused by motion artifacts that exceed the displacement of feature points. In this study, we developed a personalized and customized facial expression recognition (FER) model that can monitor changes in a person’s internal state by recognizing the transitions in their daily facial expressions. The model employs image comparison to monitor a specific person. It is tailored to each individual by training it with the individual’s actual facial expressions. This model is not universal but can be adjusted to individual emotional nuances. To account for arbitrary transitions along positive–negative facial expressions, we developed a two-stage FER framework comprising three deep neural networks (DNNs). The proposed framework uses images without detecting facial feature points. The first stage performs binary classification on the input image pairs and categorizes each image as positive or negative. The second stage compares the images to predict the direction of changes, i.e., whether the expression shifts toward positive or negative. This stage is trained on transitional image pairs, focusing on neutral–positive, neutral–negative, and positive–negative changes, respectively. To validate the proposed framework, we collected datasets that include subtle transitions within and between positive and negative facial expressions, trained the networks, and evaluated their performance. The results demonstrate that the proposed two-stage framework outperforms conventional FER frameworks on datasets consisting of both positive and negative facial expression transitions.