<p>We present a unified geometric framework for modeling learning dynamics in physical, biological, and machine learning systems. The theory reveals three fundamental regimes, each emerging from the power-law relationship <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\( g \propto \kappa ^\alpha \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>g</mi> <mo>∝</mo> <msup> <mi>κ</mi> <mi>α</mi> </msup> </mrow> </math></EquationSource> </InlineEquation> between the metric tensor <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\( g \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>g</mi> </math></EquationSource> </InlineEquation> in the space of trainable variables and the noise covariance matrix <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\( \kappa \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>κ</mi> </math></EquationSource> </InlineEquation>. The quantum regime corresponds to <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\( \alpha = 1 \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>α</mi> <mo>=</mo> <mn>1</mn> </mrow> </math></EquationSource> </InlineEquation> and describes Schrödinger-like dynamics that emerges from a discrete shift symmetry. The efficient learning regime corresponds to <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\( \alpha = \tfrac{1}{2} \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>α</mi> <mo>=</mo> <mstyle displaystyle="false" scriptlevel="0"> <mfrac> <mn>1</mn> <mn>2</mn> </mfrac> </mstyle> </mrow> </math></EquationSource> </InlineEquation> and describes very fast machine learning algorithms. The equilibration regime corresponds to <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\( \alpha = 0 \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>α</mi> <mo>=</mo> <mn>0</mn> </mrow> </math></EquationSource> </InlineEquation> and describes classical models of biological evolution. We argue that the emergence of the intermediate regime <InlineEquation ID="IEq7"> <EquationSource Format="TEX">\( \alpha = \tfrac{1}{2} \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>α</mi> <mo>=</mo> <mstyle displaystyle="false" scriptlevel="0"> <mfrac> <mn>1</mn> <mn>2</mn> </mfrac> </mstyle> </mrow> </math></EquationSource> </InlineEquation> is a key mechanism underlying the emergence of biological complexity.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Geometric Learning Dynamics

  • Vitaly Vanchurin

摘要

We present a unified geometric framework for modeling learning dynamics in physical, biological, and machine learning systems. The theory reveals three fundamental regimes, each emerging from the power-law relationship \( g \propto \kappa ^\alpha \) g κ α between the metric tensor \( g \) g in the space of trainable variables and the noise covariance matrix \( \kappa \) κ . The quantum regime corresponds to \( \alpha = 1 \) α = 1 and describes Schrödinger-like dynamics that emerges from a discrete shift symmetry. The efficient learning regime corresponds to \( \alpha = \tfrac{1}{2} \) α = 1 2 and describes very fast machine learning algorithms. The equilibration regime corresponds to \( \alpha = 0 \) α = 0 and describes classical models of biological evolution. We argue that the emergence of the intermediate regime \( \alpha = \tfrac{1}{2} \) α = 1 2 is a key mechanism underlying the emergence of biological complexity.