Robust Simplex-Based Multinomial Logistic Regression
摘要
The multicategory logistic regression (MLR) is one of the most popular large-margin classifiers in machine learning. Although it has been successfully applied in various fields, some intrinsic problems still remain. In particular, the existing MLR models suffer from the over-specification of decision functions or the choice of reference category and are very sensitive to the potential outliers due to the unbounded loss function. In this article, utilizing the prevalent simplex-based framework, we propose three robust MLR models based on truncated loss function, weighted learning and label-adjusted learning, respectively. Moreover, the first two approaches achieve robustness by removing potential outliers and the latter obtains robustness by adaptively relabeling outliers. Theoretical properties including Fisher consistency, probability estimation and breakdown point are well established. Intensive numerical studies demonstrate that the proposed methods are very competitive for problems with potential outliers.