Scholarlink

<p>Automatic image captioning systems for chest X-ray images have attracted the attention of scientists and radiologists and are increasingly being integrated into software systems in hospitals. Traditionally, the chest X-ray image captioning methods face several challenges, particularly regarding data size and medical terminologies. These methods often use supervised multi-label classification models to identify medical terms and detect clinical signs in images. However, understanding the relationship among these clinical signs in the image and with medical term is very important to create accurate captions. In this study, we propose a model, called VTG-Transformer, which utilizes the Transformer architectures to analyze images and generate captions, while leveraging the ability to predict and encode clinical signs. It can generate simultaneously captions and medical terms including clinical signs and their relationships. The difference between generating captions from images combined with tags and other caption-generation methods is that the tags associated with the image are identified and then they are used as conditions to generate more accurate captions. The method not only generates captions but also interprets the relationship between clinical signs and image components, which previous methods have not attempted. Evaluating on two benchmark datasets shows that our model excels in natural language generation (NLG) measurements, with impressive results on the ad-hoc dataset. The proposed model can combine image features and recognize tags to generate captions. The experimental results have shown its promising performance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Novel Transformer-Based Framework for Chest X-Ray Captioning with Clinical Sign Detection

  • 期刊论文

  • 发表日期：

    2025年6月4日

查看全文
  • Thanh T. Tran,
  • Duy V. M. Nguyen,
  • Hieu Trung Huynh

摘要

Automatic image captioning systems for chest X-ray images have attracted the attention of scientists and radiologists and are increasingly being integrated into software systems in hospitals. Traditionally, the chest X-ray image captioning methods face several challenges, particularly regarding data size and medical terminologies. These methods often use supervised multi-label classification models to identify medical terms and detect clinical signs in images. However, understanding the relationship among these clinical signs in the image and with medical term is very important to create accurate captions. In this study, we propose a model, called VTG-Transformer, which utilizes the Transformer architectures to analyze images and generate captions, while leveraging the ability to predict and encode clinical signs. It can generate simultaneously captions and medical terms including clinical signs and their relationships. The difference between generating captions from images combined with tags and other caption-generation methods is that the tags associated with the image are identified and then they are used as conditions to generate more accurate captions. The method not only generates captions but also interprets the relationship between clinical signs and image components, which previous methods have not attempted. Evaluating on two benchmark datasets shows that our model excels in natural language generation (NLG) measurements, with impressive results on the ad-hoc dataset. The proposed model can combine image features and recognize tags to generate captions. The experimental results have shown its promising performance.