This work presents a modular training approach for deep neural networks applied to text classification, introducing the concept of a sector as a trainable layer and its subsequent non-trainable layers up to the next trainable layer. Each sector is trained independently using auxiliary models that mimic the original network’s output layer, allowing for training without full end-to-end backpropagation. The method was evaluated on 1D ConvNet, Transformer, and bidirectional LSTM architectures across five benchmark text classification data sets, using 1%, 5%, and 10% of the data for training. Results show that sector-wise modular training achieves comparable or even superior accuracy to traditional end-to-end training while reducing computational time, particularly in four out of five data sets. This approach offers a generalizable alternative to standard training methods in deep learning for text classification.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Sector-Wise Backpropagation for Low-Resource Text Classification in Deep Models

  • José Luis Vázquez Noguera,
  • Carlos U. Valdez,
  • Marvin M. Agüero,
  • Julio C. Mello,
  • José D. Colbes,
  • Sebastián A. Grillo

摘要

This work presents a modular training approach for deep neural networks applied to text classification, introducing the concept of a sector as a trainable layer and its subsequent non-trainable layers up to the next trainable layer. Each sector is trained independently using auxiliary models that mimic the original network’s output layer, allowing for training without full end-to-end backpropagation. The method was evaluated on 1D ConvNet, Transformer, and bidirectional LSTM architectures across five benchmark text classification data sets, using 1%, 5%, and 10% of the data for training. Results show that sector-wise modular training achieves comparable or even superior accuracy to traditional end-to-end training while reducing computational time, particularly in four out of five data sets. This approach offers a generalizable alternative to standard training methods in deep learning for text classification.