Long-text mathematical word problems (MWPs) offer a stringent probe of large-language-model (LLM) reasoning, yet reliable benchmarks scarcely exist because hand-crafting rich, distractor-laden narratives is labor-intensive. We present an automatic short-to-long pipeline that transforms abundant short MWPs (e.g., GSM8K) into context-heavy variants while rigorously preserving solution correctness. The IEC pipeline operates in three successive stages: first, an Interference step inserts answer-neutral numeric distractors; next, an Extension step uses GPT-4-turbo to expand the now-noisy prompt into a much longer narrative while preserving all original conditions; finally, an automated Check stage filters any draft that leaks reasoning steps or alters the correct answer, regenerating up to three times before rolling back to the previous round. Applying IEC pipeline to 1318 GSM8K items yields a corpus whose average length surpasses 2000 tokens—over 30 times the original—while achieving an 85% validity rate. We then benchmark three representative LLMs—DeepSeek-chat, GLM-4-plus, and Kimi—across four expansion depths. All excel on the short source set (above 0.93 accuracy) but degrade as length increases, with Kimi dropping to 0.39 after six rounds, underscoring persistent long-context weaknesses. Our results establish a scalable route to high-quality long-text MWP datasets and reveal concrete failure modes that future model architectures, training curricula, and retrieval/memory mechanisms must address.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Long Math Reasoning Problem Generation

  • Changwei Li,
  • Guangping Huang,
  • Zihao Zhou,
  • Qiufeng Wang

摘要

Long-text mathematical word problems (MWPs) offer a stringent probe of large-language-model (LLM) reasoning, yet reliable benchmarks scarcely exist because hand-crafting rich, distractor-laden narratives is labor-intensive. We present an automatic short-to-long pipeline that transforms abundant short MWPs (e.g., GSM8K) into context-heavy variants while rigorously preserving solution correctness. The IEC pipeline operates in three successive stages: first, an Interference step inserts answer-neutral numeric distractors; next, an Extension step uses GPT-4-turbo to expand the now-noisy prompt into a much longer narrative while preserving all original conditions; finally, an automated Check stage filters any draft that leaks reasoning steps or alters the correct answer, regenerating up to three times before rolling back to the previous round. Applying IEC pipeline to 1318 GSM8K items yields a corpus whose average length surpasses 2000 tokens—over 30 times the original—while achieving an 85% validity rate. We then benchmark three representative LLMs—DeepSeek-chat, GLM-4-plus, and Kimi—across four expansion depths. All excel on the short source set (above 0.93 accuracy) but degrade as length increases, with Kimi dropping to 0.39 after six rounds, underscoring persistent long-context weaknesses. Our results establish a scalable route to high-quality long-text MWP datasets and reveal concrete failure modes that future model architectures, training curricula, and retrieval/memory mechanisms must address.