We prove global well-posedness and scattering for solutions to the mass-critical inhomogeneous nonlinear Schrödinger equation \(i\partial _{t}u+\Delta u=\pm |x|^{-b}|u|^{\frac{4-2b}{d}}u\) for large \(L^2({\mathbb {R}} ^d)\) initial data with \(d\ge 3,0<b<\min \left\{ 2,\frac{d}{2} \right\} ;\) in the focusing case, we require that the mass is strictly less than that of the ground state. Compared with the classical Schrödinger case \((b=0,\) Dodson in J Am Math Soc 25:429–463, 2012, Adv. Math. 285:1589–1618, 2015), the main differences for the inhomogeneous case \((b>0)\) are that the presence of the inhomogeneity \(|x|^{-b}\) creates a nontrivial singularity at the origin, and breaks the translation symmetry as well as the Galilean invariance of the equation, which makes the establishment of the profile decomposition and long time Strichartz estimates more difficult. To overcome these difficulties, we perform the concentration compactness/rigidity methods of Kenig and Merle (Invent Math 166:645–675, 2006) in the Lorentz space framework, and reduces the problem to the exclusion of almost periodic solutions. The exclusion of these solutions will utilize fractional estimates and long time Strichartz estimates in Lorentz spaces. In our study, we observe that the decay of the inhomogeneity \(|x|^{-b}\) at infinity prevents the concentration of the almost periodic solution at infinity in either physical or frequency space. Therefore, we can use classical Morawetz estimates, rather than interaction Morawetz estimates, to exclude the existence of the quasi-soliton. Moreover, our method can also be applied to the case \( b = 0 ,\) thereby recovering the scattering results in Tao et al. (Duke Math J 140:165–202, 2007) and Killip et al. (Anal PDE 1:229–266, 2008).