Solving separable nonlinear least squares problems with multiple datasets

In 1978 Golub and LeVeque considered an exponential fitting problem with multiple datasets where the nonlinear variables, e.g., the decay rates, had to hold for all the datasets simultaneously, but the linear variables, e.g., the pre-exponentials, could vary from one dataset to the next. They showed that with the variable projection technique, one could reduce the problem to only the nonlinear variables. Golub and LeVeque also showed that the main matrix of the algorithm was block diagonal with the same matrix down the diagonal. This allowed them to compute a solution while storing only the main matrix associated with a single dataset, so that the memory requirements of the problem are independent of the number of datasets. Since then, papers using this observation have appeared in the biophysics literature, in the systems identification literature, in the medical literature for studying disease of the retina, in the spectroscopy literature, and in the numerical analysis literature for determining the knots in a 2 dimensional spline problem. In 2007 the TIMP package was created in the statistical language R by Mullen and van Stokkum to handle spectroscopy problems which might have as many as 1000 datasets. The TIMP package, which handles several models, uses finite differences to approximate derivatives. In this paper we show that by using a tensor product of orthogonal matrices, the number of rows for the Jacobian for the multiple dataset problem can be significantly reduced.

Keywords: Separable nonlinear least squares; multiple data sets; variable projections