How to Parallelize ‘‘Non-Parallelizable’’ Minimization Functions
摘要
The paper proposes an algorithm for parallelizing calculations that arise when using highly optimized minimization functions available in many computing packages. The main idea of the proposed algorithm is based on the fact that although the ‘‘inner workings’’ of the minimization function used may not be known to the user, it inevitably uses in its work auxiliary functions that implement the calculation of the minimized functional and its gradient, which are usually realized by the user and in many cases can be parallelized. The paper discusses in detail both the parallelization algorithm and its software implementation using MPI parallel programming technology, which can act as template for parallelizing a wide set of applied minimization problems. An example of software implementation of the proposed algorithm is demonstrated using the Python programming language, but can be easily rewritten using the C/C++/Fortran programming languages.