In high-dimensional regression, the choice of regularization penalty typically forces a rigid assumption upon the underlying signal structure, dichotomizing data into strictly sparse (Lasso, \(q=1\) ) or entirely dense (Ridge, \(q=2\) ) regimes. However, real-world data generating mechanisms frequently exist on a continuum between these extremes, requiring flexible geometries to handle varying degrees of sparsity and considerable multicollinearity. In this work, we propose a data-driven framework to learn the optimal regularization norm by elevating the \(L_q\) exponent ( \(q \in (0, 2]\) ) from a discrete choice to a strictly continuous, learnable hyper-parameter. To overcome the computational bottleneck of evaluating non-convex and non-smooth penalty landscapes, we develop a universal proximal coordinate descent solver that utilizes a safeguarded jumping threshold operator and a novel empirical Karush-Kuhn-Tucker (KKT) verification strategy. This solver is coupled with a stochastic Tree-structured Parzen Estimator (TPE) utilizing randomized internal validation splits, enabling the rapid discovery of optimal penalty geometries without over-fitting. We evaluate the framework on simulated architectures, demonstrating its dynamic adaptivity to structural sparsity, collinearity, and varying signal-to-noise ratios. Applied to four high-dimensional genomic datasets (scaling up to \(P \approx 50,000\) features), our generalized adaptive bridge regression (GABR) framework successfully identifies optimal, off-grid grouping architectures ( \(q \approx 1.63\) to 1.80), outperforming purely sparse and purely dense alternatives. These results demonstrate that the exact regression geometry can be efficiently learned from the data, enabling a unified approach to high-dimensional inference without the computational restrictions of exhaustive discrete grid searches.
