Deep learning models for time series classification, such as 1D Convolutional Neural Networks, are increasingly being used to predict the outcome of time-dependent clinical data, such as electrocardiograms (ECGs) and electroencephalograms (EEGs). However, when dealing with such complex model architectures, which behave as opaque boxes, the interpretability of new predictions is lost. Explainable artificial intelligence (XAI) techniques aim to mitigate this issue by enabling some degree of explanation for the model output, given a new input. In the case of biosignals, represented by multivariate time series, it is desirable to indicate which dimensions and which segments led the model to the final outcome. In this context, this work aims to evaluate and compare the use of different XAI methods, gradient-based or perturbation-based, to better understand the predictions produced by deep learning models in biosignals classification tasks. The experimental results with a synthetic dataset and a real-world ECG dataset indicate that perturbation-based approaches are the best suited for faithful and robust explanations, while gradient-based methods achieve similar results, but are more cost-efficient.