Stress detection has lately acquired importance, and crucial roles it plays in mental and physical health management. Detection of stress in an individual is very much important as early and accurate determination can avoid serious health problems like anxiety, depression, and cardiovascular diseases in old age. The methods followed have generally been based on such subjective assessments, which may inaccurately be biased. It has become possible nowadays to develop computerized systems of stress detection based on analysis of physiological data, although with much more objectivity, since recent breakthroughs have occurred in the fields of machine learning and deep learning. This study emphasizes hybrid deep learning models, which can, by combining several architectures like CNNs and LSTMs, reach high accuracy in stress detection. However, such “black-box” models often cause huge concern about their interpretability in sensitive applications such as healthcare. In this direction, XAI techniques like SHAP and LIME are very effectively incorporated in such frameworks for insights into model decisions. The paper gives a broad overview of the studies done up to date and compares different methods, datasets, and performance metrics that have advantages and limitations over different approaches for hybrid models combined with XAI as an avenue to be considered for future work in developing accurate, interpretable, and user-friendly stress detection systems.