Recent text-conditioned image generation models have demonstrated an exceptional capacity to produce diverse and creative imagery with high visual quality. However, when trained on billion-scale datasets randomly collected from the Internet, where human aesthetic preferences are inadequately learned, these models often generate facial images that deviate from mainstream aesthetics, particularly across different racial groups. While some existing methods attempt to address this issue by fine-tuning models on large-scale, manually annotated facial datasets, such approaches incur substantial annotation and computational costs. To overcome these limitations, we propose a framework called FP-Director, which learns a facial-preference direction in the latent space and updates latent codes to refine the generated results. This alignment process is applied during inference and does not require fine-tuning of the original model or access to large-scale datasets. Extensive empirical evaluations demonstrate that FP-Director significantly improves both overall quality and aesthetics of generated faces.