Evaluating Prompt Strategies for Generative AI-Based Readability Improvement of Pillar 3 Disclosures
摘要
Pillar 3 disclosures are critical regulatory documents intended to inform the public and promote transparency. However, their technical complexity often renders them inaccessible to non-expert audiences. This study evaluates the use of Generative Artificial Intelligence (GenAI) models to improve the readability of these financial documents without omitting key information. Two state-of-the-art GenAI models, GPT-4o and GPTo3, were tested with six prompt strategies, including standardized, enriched, roleplay, and Chain-of-Thought approaches. Readability was measured using the Gunning Fog Index and LIX metrics, and results were further examined using residual analysis and human review for financial accuracy and tone. All prompts significantly reduced text complexity, with the Chain-of-Thought prompt achieving the most substantial improvement, particularly for GPTo3. However, GPT-4o occasionally failed to complete outputs and showed higher variability in quality. Human evaluation confirmed that while GenAI models preserved most key financial information, some prompts led to oversimplification or informal wording. The results show that Chain-of-Thought prompt improved the readability by an average 12.30 for Gunning Fog Index and 37.99 for LIX, with statistically significant improvements across all prompts and models. These findings highlight the potential of GenAI to assist in simplifying financial disclosures, provided human oversight and further prompt design optimization are maintained.