Multi-label learning focuses on a special classification problem where any instance could belong to multiple class labels simultaneously. Thus its corresponding label matrix is low-rank and compressible, which makes label space dimensionality reduction possible, via label embedding and selection. In this paper, we will pay more attention on a single-stage label selection where a selected label subset and its corresponding recovery matrix are created once. Interpolative matrix decomposition (IMD) approximates a primary matrix with a product of two low-rank matrices, where its left matrix is a column subset of original matrix and its right matrix acts as a recovery matrix. This mathematical formulation just describes our label selection problem. We build a joint matrix with both feature and label matrices, which is factorized using IMD, resulting in a feature-aware label selection algorithm or LS-IMDf simply. To the best of our knowledge, our LS-IMDf is the first feature-aware single-stage label selection technique. Its simplified version is LS-IMD without feature information. Finally, we validate the effectiveness of our LS-IMDf empirically on three benchmark data sets with 100+ labels, via comparing with five existing methods and our LS-IMD, according to two evaluation metrics (Precision and DCG, n=1, 3 and 5).