The study addresses the critical challenge of improving downlink performance, namely average throughput, latency, and packet loss in ultra-dense 6G networks. The issue intensifies when the number of users increases and traditional resource allocation methods fail under high-bandwidth (BW) mmWave settings. To solve this, a novel framework that integrates massive MIMO (mMIMO), Cognitive Radio (CR), and Reconfigurable Intelligent Surfaces (RIS) with Power-Domain (PD) NOMA was proposed. The approach includes a Hierarchical Reinforcement Learning (HRL) algorithm for adaptive power allocation, leveraging a logarithmically-scaled reward function to promote training stability. The system was tested using several Monte Carlo simulations in a 30 GHz mmWave environment with 512-QAM modulation. This study utilized bespoke simulations in MATLAB, including realistic channel models to authenticate the proposed system. The results showed that the HRL-optimized combined mMIMO-CR-RIS system worked much better than the baseline setups. At a user density of 100 and a signal-to-noise ratio of 30 dB, the HRL-driven solution produced a peak average throughput of 4.99 Gbps, a latency of 2.91 ms, and a packet loss rate of 4.67%. These numbers are 18.5%, 32.6%, and 6.16% better than non-HRL benchmarks, respectively. The synergistic combination of RIS and CR, dynamically regulated by HRL is a profoundly successful technique for minimizing interference and boosting QoS in 6G networks.