Interaction Modeling in Logistic Regression: An Analysis of Dropout Regularization and Adaptive Optimization
摘要
The introduction of regularization techniques and neural networks in the field of Machine LearningMachine learning has garnered substantial research. This provides sufficient knowledge on the effects of dropoutsDropout and optimizersOptimizer in the case of classical neural networks. However, the consequences of the same using logistic regressionLogistic regression, across various interaction degreesInteraction degree, haven’t been thoroughly studied. In this paper, we aim to discuss the effects of dropoutsDropout with different optimizersOptimizer (of different values for the hyperparameters—momentumMomentum and dropoutDropout probability) on interaction degreesInteraction degree varying from 1 to 3 and demonstrate, both mathematically and empirically, that the best performance for a logistic regressionLogistic regression model is a combination of Adam optimizerOptimizer and Gaussian dropoutsDropout with Interaction degreeInteraction degree = 2.