Scholarlink

<p>We propose <i>Logistic Map Dropout</i>, a novel regularization mechanism that leverages the chaotic dynamics of the logistic map to perform continuous, stateful dropout in deep neural networks. Unlike standard dropout which stochastically zeros out activations, our method deterministically scales activations by a value in [0,&#xa0;1] derived from the logistic map, enabling smooth attenuation of activations and uninterrupted gradient flow. To bridge the gap between continuous and binary regularization, we introduce two extensions: <i>Sparse Scaling</i>, which thresholds the logistic map to induce sparsity while preserving strong activations, and <i>Modulated Sparse Scaling</i>, which allows selective zeroing to approximate classical dropout behavior. Extensive empirical evaluations across image classification, language modeling, and graph prediction tasks demonstrate that our method achieves competitive or superior generalization compared to standard and structured dropout baselines. In particular, modulated variants consistently outperform stochastic regularizers on LSTM and GNN architectures, highlighting the benefit of chaos-driven deterministic masking. Logistic Map Dropout offers a principled alternative to stochastic regularization by blending chaotic dynamics with structured noise, yielding a tunable spectrum between soft and hard dropout regimes. This work opens a new direction for applying dynamical systems theory in neural network regularization.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Logistic Dropout: a soft dropout mechanism via the Logistic Map

  • 期刊论文

  • 发表日期：

    2026年1月12日

查看全文
  • Kenneth Brezinski,
  • Ken Ferens,
  • Witold Kinsner

摘要

We propose Logistic Map Dropout, a novel regularization mechanism that leverages the chaotic dynamics of the logistic map to perform continuous, stateful dropout in deep neural networks. Unlike standard dropout which stochastically zeros out activations, our method deterministically scales activations by a value in [0, 1] derived from the logistic map, enabling smooth attenuation of activations and uninterrupted gradient flow. To bridge the gap between continuous and binary regularization, we introduce two extensions: Sparse Scaling, which thresholds the logistic map to induce sparsity while preserving strong activations, and Modulated Sparse Scaling, which allows selective zeroing to approximate classical dropout behavior. Extensive empirical evaluations across image classification, language modeling, and graph prediction tasks demonstrate that our method achieves competitive or superior generalization compared to standard and structured dropout baselines. In particular, modulated variants consistently outperform stochastic regularizers on LSTM and GNN architectures, highlighting the benefit of chaos-driven deterministic masking. Logistic Map Dropout offers a principled alternative to stochastic regularization by blending chaotic dynamics with structured noise, yielding a tunable spectrum between soft and hard dropout regimes. This work opens a new direction for applying dynamical systems theory in neural network regularization.