Scholarlink

<p>The recent explosive development of artificial intelligence is largely driven by deep neural networks such as Transformers. Computational capabilities of deep neural networks have advanced rapidly, and their applications have expanded across a wide range of scientific fields. Motivated by the precise computation of mutual information by deep neural networks, named mutual information neural estimation (MINE), we investigate the relationship between the rate function in the large deviation principle and the Kullback–Leibler divergence, and prove an equality between them. We further cast an interpretation of MINE in terms of the rate function.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Some comments on the relationship between the rate function in the large deviation principle and the Kullback–Leibler divergence: toward the interpretation of neural estimation of mutual information

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月31日

查看全文
  • Ichiro Tsuda,
  • Takao Namiki

摘要

The recent explosive development of artificial intelligence is largely driven by deep neural networks such as Transformers. Computational capabilities of deep neural networks have advanced rapidly, and their applications have expanded across a wide range of scientific fields. Motivated by the precise computation of mutual information by deep neural networks, named mutual information neural estimation (MINE), we investigate the relationship between the rate function in the large deviation principle and the Kullback–Leibler divergence, and prove an equality between them. We further cast an interpretation of MINE in terms of the rate function.