An increasing body of literature on clinical decision support systems (CDSSs) utilizing deep learning (DL) mostly focusses on direct comparisons between CDSSs and physicians (human versus computer). This paper examines the feasibility of techniques of explainable neural networks for decision-making assistance in healthcare text analysis scenarios. We employed a clear methodology on the same medical dataset to enhance the accessibility of the judgements made by Convolutional Neural Networks (CNN). The objective is to enhance health experts’ trust regarding box forecasts. We employed understandable deep learning techniques, specifically Local Interpretable Model-agnostic Descriptions (LIME) and Compound Arguments, in conjunction with a different interpretive structure known as the Situational Important and Utility (CIU) approach. Individuals from several non-medical disciplines conducted a series of assessments in an internet-based questionnaire format and recorded their opinions and comprehension of the provided justifications. Three user categories (N = 40, 40, 40) underwent quantitative analysis, each presented with three different types of reasons. Our findings validate the prediction that the CIU-explainable technique surpassed both LIME and THAP strategies in facilitating decision-making support while also being more accessible and comprehensible to users. Moreover, CIU surpassed the LIME algorithm and THAP by producing justifications more swiftly. Therefore, we suggest three feasible transparent techniques that, with potential modifications in execution, can be tailored to diverse clinical information and furnish superior decision-making support for healthcare practitioners.