Logical Reasoning plays an important role in machine reading comprehension (MRC) tasks by enabling the recognition of hidden connectives and relations in text. However, previous studies still struggle to recognize logical structures in text and do not fully utilize the relationship between textual units in these structures. In this paper, we propose LogiDRS, a novel approach that leverages the sense of relationship between textual units to predict an answer to tackle this challenge. Specifically, our method extracts Elementary Discourse Units (EDUs) and employs a pre-trained discourse relation sense classifier to identify the semantic relationships between these units. This information is then integrated into the embeddings of connectives, enriching their representation. Subsequently, the complete sequence of EDUs and connectives, now enhanced with the discourse relation sense, is processed by a Transformer-based encoder to focus on relevant discourse structures before making a prediction. We conduct experiments on two benchmark MRC datasets: LogiQA, Reclor and achieve competitive results in both datasets.