A Facial Expression Generation Based on Custom Emotional Language and Foundation Models
摘要
Reasoning about facial expressions is now simpler, yet generating expressions that reflect context and memory remains challenging. We propose a facial expression generation algorithm leveraging a custom emotional language and a foundation model. The emotional language encodes states as base-27 letters, allowing representation of nuances across time slices. Each “word” stands for a momentary emotion, and transitions form “sentences.” This approach leverages pretrained models like ChatGPT to generate symbolic expressions without extensive retraining, or a fine-tuned model like LLaMAv2 for tighter integration. We validate using the CANDOR corpus, which consists of over 1,000 h of human-human video interactions. Facial expressions are stored as a seven-dimensional vector that sums to one, then converted into base-27 words resembling standard text. Cosine similarity compares generated expressions against ground truth to quantify accuracy. Though reasoning with large models can be slow, we plan to refine real-time pipelines and test with human participants through a robotic head interface. Handcrafted questionnaires will further evaluate our system’s ability to authentically convey and respond with nuanced emotional states. Ultimately, this integration of emotional language into foundation models aims to provide more natural human-like interactions, bridging the gap between simple recognition and genuine emotive expression. Tests will refine results.