ViBBQ: Probing Social Biases in Vietnamese LLMs via Translated and Culturally Extended Benchmarks
摘要
While benchmarks like BBQ have established standards for evaluating social biases in Large Language Models (LLMs), relying solely on translated datasets creates a “safety mirage” for low-resource languages. In this paper, we introduce ViBBQ, a comprehensive benchmark for probing social biases in Vietnamese LLMs, comprising over 61,000 samples constructed through a hybrid pipeline of translation and agentic cultural extension. Our evaluation of five state-of-the-art open-source models reveals a critical “Bias Reversal” phenomenon: while models exhibit aggressive safety over-correction (negative bias) on translated Western contexts, they revert to strong pro-stereotypical prejudices (positive bias) on Vietnam-specific axes generated from local news. These findings confirm that current safety alignment mechanisms are culturally conditioned effective against globalized concepts but permeable to indigenous biases. Among evaluated models, Llama-3.1-8B demonstrates the most robust performance, yet the pervasive inconsistency across contexts underscores the urgent need for culturally grounded safety alignment in Southeast Asian languages.