We present a novel linear privacy-preserving machine learning framework that extends the standard LDP-SGD algorithm by incorporating dynamic self-feedback privacy budget allocation. During each iteration, the algorithm reallocates a larger share of the budget to coordinates whose gradients are most informative, as inferred from the optimizer’s own per-coordinate statistics. This fully internal feedback loop eliminates extra data queries and incurs no additional privacy loss, unlike earlier adaptive methods that rely on pre-computed, static feature correlations. A warm-up phase stabilizes the statistics at the start of training. By replacing the uniform per-attribute budget of standard LDP-SGD, the framework also remedies the sharp accuracy drop traditionally observed on small datasets under tight privacy constraints. Experiments on benchmark small-scale datasets show average and peak accuracy gains of 8.1% and 13.9%, respectively, relative to standard LDP-SGD at low privacy budgets.