Diffusion models have excelled in addressing a variety of inverse problems. Nevertheless, their application is restricted by the requirement for specific prior knowledge of the forward operator. This paper presents a novel approach, UFODM, which circumvents this constraint by selecting the appropriate forward measurement, making the method more applicable to real-world scenarios. Specifically, our approach enables the concurrent estimation of both the reconstructed image and the characteristics of the forward operator during the inference stage. Our method effectively tackles inverse problems such as blind deconvolution, JPEG restoration, and super-resolution. Furthermore, we demonstrate the versatility of our approach in solving generic inverse problems through the automated selection of forward operators. Empirical evidence suggests that our framework has the potential to enhance the efficacy of diffusion models and extend their applicability in solving real-world inverse problems.