We study sparse parameter identification for stochastic systems under general observation sequences, where accurate estimation of nonzero parameters is critical, as estimation errors can carry over to downstream risk-sensitive decisions. Existing non-i.i.d. sparse identification schemes largely rely on \(L_p\) -type regularization, where persistent marginal penalties systematically shrink nonzero coefficients toward zero. To address this limitation, we consider a sparse identification framework based on a class of nearly unbiased regularizers and highlight the vanishing-derivative property for large coefficients as the structural mechanism that mitigates shrinkage on nonzero coefficients. Within this framework, without requiring i.i.d. or stationarity assumptions on the observation sequences, we establish almost-sure convergence and finite-time support recovery, and show that the vanishing-derivative structure yields asymptotic normality for nonzero coefficients. The proposed approach is applied to closed-loop STR identification and sparse volatility forecasting with exogenous factors. Experiments on simulations and high-frequency cryptocurrency data demonstrate the effectiveness of the proposed method.
