This investigation examines instability mechanisms and turbulent transition in the stationary disk boundary layer of a rotor-stator cavity through an integrated approach combining two-dimensional direct numerical simulations (2D DNS), global linear stability analysis, and three-dimensional direct numerical simulations (3D DNS). The base flow characteristics, the global linear stability properties, and the nonlinear evolution processes leading to turbulence are explored. The research examines an annular, enclosed rotor-stator cavity with curvature parameter \(Rm = (b^*+a^*)/(b^*-a^*)=1.8\) , and an aspect ratio \(L=(b^*-a^*)/(2h^*) = 5\) , where \(a^*\) and \(b^*\) are the minimum and maximum radius, respectively, and \(2h^*\) denotes the inter-disk spacing. The current study examines Reynolds numbers \(Re_\varphi = \Omega ^*_db^{*2}/\nu ^*\) ranging from \(10^4\) to \(10^5\) , where \(\Omega ^*_d\) is the angular velocity of the rotor, \(\nu ^*\) is the fluid’s kinematic viscosity. The results reveal that for \(Re_\varphi \le 3.6 \times 10^4\) , the base flow converges to a stable equilibrium state. However, when \(Re_\varphi \) exceeds \(3.7 \times 10^4\) , the base flow destabilizes, giving rise to persistent velocity oscillations within the stationary disk boundary layer. Global linear stability analysis confirms these observations. For axisymmetric disturbances (azimuthal wavenumber \(\beta = 0\) ), the critical Reynolds number for global linear instability is \(Re_\varphi = 3.67 \times 10^4\) , precisely demarcating the threshold for self-sustained oscillations in the base flow. For non-axisymmetric disturbances with \(\beta = 15\) , a more complex dynamics is identified: the maximum linear temporal growth rate occurs at \(Re_\varphi = 2.5 \times 10^4\) , while the critical threshold for global linear instability emerges at a lower \(Re_\varphi = 1.58 \times 10^4\) , marking the onset conditions for a self-sustained spiral wave mode. The 3D DNS results reveal sophisticated nonlinear dynamics where spiral wave energy is transferred from higher to lower radial positions through complex mode interactions. When circular waves dominate in the flow field, their interaction with spiral waves across various radial positions generates a rich spectral composition with multiple distinct dominant frequencies within the boundary layer. At \(Re_\varphi \ge 7.0 \times 10^4\) , the flow transitions to a chaotic state with turbulent characteristics. Remarkably, even in this strongly chaotic regime, coherent circular wave structures remain observable within the chaotic flow field, although distinct dominant frequencies are no longer present in the spectral analysis.