Many real-world data, such as chemical compounds and proteins, are naturally modelled as datasets of graphs. However, the labels of the graphs are often difficult to obtain due to the high relative cost to label graphs. To extract knowledge from a dataset of unlabelled graphs, we aim to conduct the task of interpretable graph-level clustering, which aims to find good clusters of graphs and also gain useful insights into the clustering result by interpreting why each graph is allocated to its corresponding cluster. To the best of our knowledge, this is a novel task that has not been systematically studied in the literature. In this paper, we successfully tackle this task by developing an interpretable deep graph-level clustering (IDGC) framework, which not only achieves good clustering performance, but also provides insightful interpretations on the clustering result. Extensive experiments on six benchmark datasets demonstrate the outstanding performance of our method. Our code is available at: https://github.com/cjbbb/IDGC-implementation .