<p>This paper presents a new model-free Q-learning approach to solve robust output consensus problems for heterogeneous multi-agent discrete-time (DT) systems. In connection with an auxiliary system, the problem is reformed to an augmented system including dynamics of agents and the leader. In this respect, we will consider a discounted performance criterion approach to obtain the solutions using RARE for developing a rigorous approach toward robust output consensus. We will also provide a lower bound on the discount factor that ensures stability of the closed-loop system, which is relevant for resilient performance against uncertainties. The Q-learning algorithm associated with the solution of RARE for an explicit model of the system's dynamics and, hence, is particularly suitable for real-world applications when such models may be unavailable or even incomplete. The proposed approach will also be proved to make the output regulation equations hold such that the tracking error converges to zero asymptotically. Comprehensive simulation examples are given to illustrate the effectiveness and robustness of the proposed approach by showing its potential to be implemented on various practical multi-agent systems.</p>