Purpose <p>In urolithiasis, hard deposits of minerals or salts are formed in the kidneys or urinary tract which may be of pure or mixed composition. It is important to know the composition of stone(s) to decide the kind of treatment and help in post-operative care to reduce re-occurrence. This study has a novel aim of selecting the best model for predicting the extent of each chemical present in the stone(s) of a patient from radiographic images (in vivo analysis).</p> Methods <p>For the stone composition analysis from images, five supervised learning models were used for multi-class-multi-output classification or multi-task classification. Multi-task classification was best suited in this case as there were seven target chemicals—calcium, magnesium, uric acid, ammonia, oxalate, phosphate, and carbonate; each of which was classified as absent, low, moderate, or high simultaneously. The metrics for such classifications are not defined in general because multiple criteria cause anomalies.</p> Results <p>In the present work, the F1 score with which each model classifies each chemical was computed, and a multi-criteria decision-making algorithm called Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) was used to find the best model. The k-nearest neighbour classifier has given the best TOPSIS performance score of 0.9034.</p> Conclusion <p>There are five different machine learning (ML) models considered in the study and were evaluated on their F1 score for each target chemical. Each of the ML classifiers was multi-class multi-output (MCMO) model, and the ultimate choice was made based on the outcome of the TOPSIS algorithm. The KNN model has found to outperform the others. This study has significance in identification and classification of stone composition from DECT images of renal calculus.</p> Graphical abstract <p></p>