The advancements in implicit neural rendering representations and differentiable rendering have facilitated the capture of multi-view RGB images from unknown illumination while simultaneously recovering the estimated object’s geometry, lighting, and materials. However, a key challenge in inverse rendering is appropriately incorporating priors and regularization during the optimization process to mitigate ill-posed situations. Currently, most methods rely on modeling lighting from different materials based on multi-view cameras using spherical Gaussians (SG), often resulting in the blurring of high-frequency details.In this study, we propose a novel inverse rendering representation called RFBR-IR. This method reduces overfitting by regularizing frequency stability during the learning process. Simultaneously, it introduces a gradient-based edge-aware factor and regularization to alleviate edge smoothness constraints in edge regions, preserving more details. Through joint optimization of radiance field, materials, and lighting, we significantly improve performance and achieve a physically-based and easily optimized inverse rendering approach. Extensive experiments demonstrate that our method outperforms state-of-the-art methods on multiple synthetic datasets, achieving superior rendering quality.