Metaheuristic algorithms, one of the sub-classes of optimization techniques, have been fast emerging as an active research area in the scientific community. These algorithms are inspired by some natural or social behaviour and result in producing efficient solutions to complex real-world optimization problems without a definitive mathematical background. One of the crucial tasks after proposing a new metaheuristic is to perform statistical analysis tests which show how significant or improved an algorithm is as compared to other similarly placed algorithms. These statistical analysis tests invariably result in the ranking of the metaheuristic algorithms for comparative analysis. In this paper, a multi-criteria tool called modified-analytical hierarchy (M-AHP) has been proposed for the ranking of six popular swarm-based metaheuristic techniques run on five classical benchmark functions. The performance has been compared with the widely used existing Wilcoxon signed-rank testing technique involving multiple comparisons in pairs and Friedman testing technique involving multiple comparisons. Based on the results achieved by the authors, the proposed method shows the same rankings as the Friedman test, but the results obtained by Wilcoxon test are deviated from the other ones.