This paper proposes a new maximization method, called as the second–derivative lower–bound function (SeLF) algorithm, which is a general principle for iteratively calculating the maximum likelihood estimate (MLE) \(\hat{\theta }\) of the parameter \(\theta \) in a one–dimensional target function \(\ell (\theta )\) (usually, the log-likelihood or marginal log-likelihood for multi-parameter cases), and its each iteration consists of two steps: A second–derivative lower–bound function step (SeLF-step) and a maximization step (M-step). The SeLF-step finds a function \(b(\theta )\) satisfying \(\ell ''(\theta )\geqslant b(\theta )\) and constructs a surrogate function \(Q(\theta |\theta ^{(t)})\) [whose form depends on \(\theta ^{(t)}\) being the t-th iteration of \(\hat{\theta }\) ] minorizing \(\ell (\theta )\) at \(\theta =\theta ^{(t)}\) . The M-step calculates the maximizer \(\theta ^{(t+1)}\) of the \(Q(\theta |\theta ^{(t)})\) function, which is equivalent to solving the equation \(\ell '(\theta ) + \int _{\theta ^{(t)}}^{\theta } b(z) \,\text {d}z =0\) to obtain its explicit solution \(\theta ^{(t+1)}\) . The SeLF algorithm holds two major advantages: (i) it strongly stably converges to the MLE \(\hat{\theta }\) , in contrast to the weakly stable convergence of minorization–maximization (MM) algorithms; and (ii) it converges regardless of initial values, in contrast to Newton’s method. The key for designing the SeLF algorithm is to find a function \(b(\theta )\) satisfying \(\ell ''(\theta )\geqslant b(\theta )\) for all \(\theta \) in the domain such that an explicit solution to the equation \(\ell '(\theta ) + \int _{\theta ^{(t)}}^{\theta } b(z) \,\text { d}z \;=\; 0\) is available. Furthermore, we develop three acceleration techniques, namely optimal SeLF, sub-optimal SeLF, and fast–SeLF algorithms, for the SeLF algorithm, resulting in a weakly stable convergence. Various applications in statistics of the proposed SeLF algorithm and three accelerated versions are introduced. The Dirichlet distribution illustrates the potential generalization of the SeLF algorithm to the multi-dimensional case. We study the convergence rates of these algorithms, accompanied by numerical experiments and comparisons.
