Scholarlink

<p>Transformers models have significantly changed many areas of Machine Learning, due to their structure and extensive number of parameters, which enables them to capture complex patterns in data. Building on this foundation, an energy-based version, the Energy Transformer (ET), has emerged as a powerful variation, achieving parameter efficiency without sacrificing performances. In this work, we introduce an innovative evolution of the ET model: the ET-KAN architecture. By integrating a Kolmogorov–Arnold Network (KAN) in place of the energy function, our model generalizes the ET as structural design, unlocking enhanced learning capabilities. We demonstrate the potential of this new architecture through an image reconstruction task, where it achieves comparable or higher results respect to the ET (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(Loss\approx 0.08\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>L</mi> <mi>o</mi> <mi>s</mi> <mi>s</mi> <mo>≈</mo> <mn>0.08</mn> </mrow> </math></EquationSource> </InlineEquation> when covering more than half of the image), outperforming the standard ET architecture while using fewer parameters. The model hereby constructed paves the way to a deeper investigation of the interplay between KAN and energy-based models, for addressing some of the key limitations of traditional transformers.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

ET-KAN: an energy-based transformer model with Kolmogorov–Arnold network for image reconstruction

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月4日

查看全文
  • Chiara Marullo,
  • Giuseppe Buonaiuto,
  • Francesco Gargiulo,
  • Massimo Esposito

摘要

Transformers models have significantly changed many areas of Machine Learning, due to their structure and extensive number of parameters, which enables them to capture complex patterns in data. Building on this foundation, an energy-based version, the Energy Transformer (ET), has emerged as a powerful variation, achieving parameter efficiency without sacrificing performances. In this work, we introduce an innovative evolution of the ET model: the ET-KAN architecture. By integrating a Kolmogorov–Arnold Network (KAN) in place of the energy function, our model generalizes the ET as structural design, unlocking enhanced learning capabilities. We demonstrate the potential of this new architecture through an image reconstruction task, where it achieves comparable or higher results respect to the ET ( \(Loss\approx 0.08\) L o s s ≈ 0.08 when covering more than half of the image), outperforming the standard ET architecture while using fewer parameters. The model hereby constructed paves the way to a deeper investigation of the interplay between KAN and energy-based models, for addressing some of the key limitations of traditional transformers.