<p>We study the effect of a vanishing noise on the phase space structure of three types of deterministic discrete systems: random Boolean networks, the Hopfield model with deterministic update dynamics, and elementary cellular automata. Boolean networks are used mainly for illustrating purposes, the attractor space of the Hopfield model is numerically sampled but the complete phase space of all minimal elementary cellular automata for small lattice sizes is exhaustively analyzed. Starting from the maximal entropy distribution (all configurations equiprobable), we show how the dynamics affects this distribution. We then investigate how a vanishing noise (i.e., a minimal one-cell-flip noise) alters this phase space, connecting attractors and modifying the asymptotic probability distribution. The main result is that for all investigated systems, this small noise can both increase or decrease the entropy of the asymptotic distribution.</p>