We propose a training algorithm for support vector machines based on their decision function, which is a kind of distance measure. We use such a measure to select the support vectors as well as set their Lagrange multiplier values. We compare our proposal with sequential minimal optimization (SMO) and classical quadratic optimization problem (QP) solver in terms of accuracy, precision, recall, and training time for several datasets. In general, the results are equivalent in accuracy; however, our proposal is faster than SMO and QP for all the considered datasets. Besides that, we highlight that our proposal can handle datasets with a large number of patterns in much less time when compared to SMO and classical QP solvers.